Home monitoring and control system

ABSTRACT

This application is directed to a home monitoring and control system including a doorbell installed at a door of a home, an application executable by a monitoring device, and a remote server. The doorbell has a button configured to, upon being touched, depressed or activated, wirelessly cause an audio speaker of an electronic device to sound an audible notification or a message to be sent to the monitoring device to indicate presence of a person at the door. The doorbell also has a camera configured to capture video data within a field of view. The remote server is configured to receive from the doorbell the video data and the message, and relay them to the monitoring device. The application is configured to receive and display at least a portion of the video data captured by the camera of the doorbell.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 14/430,124, filed Mar. 20, 2015, titled “WallSwitch,” which is a 371 application of PCT Application No.PCT/US2013/061021, filed Sep. 20, 2013, which claims the benefit ofpriority of U.S. Provisional Application No. 61/704,437 filed on Sep.21, 2012. Each of these patent applications is herein incorporated byreference in its entirety for all purposes.

This application is related to U.S. patent application Ser. No.15/972,648, titled “Methods and Systems for Home Monitoring andControl,” filed May 7, 2018, which is incorporated by reference in itsentirety.

TECHNICAL FIELD

This patent specification relates to a wall switch. More particularly,this patent specification relates to a wall switch comprising a dockingstation and a user-removable wall-switch head unit.

BACKGROUND

Some homes today are equipped with smart home networks to provideautomated control of devices, appliances and systems, such as heating,ventilation, and air conditioning (“HVAC”) system, lighting systems,alarm systems, home theater and entertainment systems. Smart homenetworks may include control panels that a person may use to inputsettings, preferences, and scheduling information that the smart homenetwork uses to provide automated control the various devices,appliances and systems in the home. For example, a person may input adesired temperature and a schedule indicating when the person is awayfrom home. The home automation system uses this information to controlthe HVAC system to heat or cool the home to the desired temperature whenthe person is home, and to conserve energy by turning offpower-consuming components of the HVAC system when the person is awayfrom the home. Also, for example, a person may input a preferrednighttime lighting scheme for watching television. In response, when theperson turns on the television at nighttime, the home automation systemautomatically adjusts the lighting in the room to the preferred scheme.

SUMMARY

Various embodiments are disclosed herein. In some embodiments, a wallswitch controller is provided that comprises a docking stationconfigured to receive a user-removable wall-switch head unit, thedocking station configured to be permanently connected to a wall andcoupled to high-power voltage wires, and a user-removable wall-switchhead unit configured to be user-insertable into said docking station anduser-removable therefrom such that the user is not exposed tohigh-voltage connections when inserting or removing. In someembodiments, the wall switch controller further comprises inputs andoutputs and circuitry for switchably controlling household line currentpower to a household electrical fixture. In some embodiments, the wallswitch controller further comprises an occupancy sensor, a temperaturesensor, or a processor.

In some embodiments, the wall switch controller further comprisescircuitry for leveraging electrical power from the household linecurrent power inputs to power the occupancy sensor, the temperaturesensor, or the processor. In some embodiments, the wall switchcontroller further comprises wireless communication circuitry configuredto communicate with a plurality of low-powered devices using a firstwireless protocol characterized by relatively low power usage andrelatively low data rates and to serve as a communications bridge to awide area network using a second wireless protocol characterized byrelatively higher power usage and relatively higher data rates, and apower storage module for storing power and using the power to maintainwireless communications during a power outage. In some embodiments, thewall switch controller further comprises a housing configured to becompatible for installation in a conventional single-gang ormultiple-gang wall box.

In some embodiments, the wall switch controller further comprises aclick-and-rotate annular ring input. In some embodiments, theclick-and-rotate annular ring input provides a menu-driven interface forcontrolling operation of the wall switch controller.

In some embodiments, the click-and-rotate annular ring input and themenu-driven interface enable remote control of a wall switch that isseparate from and communicatively coupled to the wall switch controller.In some embodiments, the click-and-rotate annular ring input enables thewall switch controller to function as a dimming light switch.

In some embodiments, a method is provided that comprises receiving, at adocking station configured to be permanently connected to a wall andcoupled to high-power voltage wires, a user-removable wall-switch headunit that is configured to be user-insertable into the docking stationand user-removable therefrom such that a user is not exposed to thehigh-voltage wires when inserting or removing; leveraging electricalpower from the high-power voltage wires to power the user-removablewall-switch head unit; and responsive to user input, controllinghousehold line current power to a household electrical fixture. In someembodiments, the method further comprises providing, by theuser-removable wall-switch head unit, a click-and-rotate annular ringinput. In some embodiments, the click-and-rotate annular ring inputreceived the user input that resulted in controlling the household linecurrent power to the household electrical fixture.

In some embodiments, the click-and-rotate annular ring input provides amenu-driven interface. In some embodiments, the click-and-rotate annularring input enables the wall switch to function as a dimming lightswitch. In some embodiments, the user-removable wall-switch head unitincludes wireless communication circuitry. In some embodiments, theuser-removable wall-switch head unit is configured to communicate with aplurality of low-powered devices using a first wireless protocolcharacterized by relatively low power usage and relatively low datarates. In some embodiments, the user-removable wall-switch head unit isconfigured to serve as a communications bridge to a wide area networkusing a second wireless protocol characterized by relatively higherpower usage and relatively higher data rates. In some embodiments, theuser-removable wall-switch head unit is configured to remotely controloperation of a second wall switch that is separate from and notphysically connected to the user-removable wall-switch head unit. Insome embodiments, the user-removable wall-switch head unit is configuredto store power and use the power to maintain wireless communicationsduring a power outage.

Various techniques for providing smart home objectives are disclosedherein. Embodiments described herein are representative examples ofdevices, methods, systems, services, and/or computer program productsthat can be used in conjunction with an extensible devices and servicesplatform that, while being particularly applicable and advantageous inthe smart home context, is generally applicable to any type of enclosureor group of enclosures (e.g., offices, factories, retail stores),vessels (e.g., automobiles, aircraft), or other resource-consumingphysical systems that will be occupied by humans or with which humanswill physically or logically interact. Thus, although particularexamples are set forth in the context of a smart home, it is to beappreciated that the scope of applicability of the described extensibledevices and services platform is not so limited.

In some embodiments, a system for accomplishing an objective in asmart-home environment, the system comprising: one or more low-powerednodes located in the smart-home environment; and one or more smart nodeslocated in the smart-home environment; the low-powered and smart nodescommunicatively interconnected for the purpose of accomplishing theobjective in the smart-home environment. At least one of the low-poweredand smart nodes is configured to: monitor a condition of the smart-homeenvironment; and transmit to at least one of the other low-powered andsmart nodes a message having information related to the condition of thesmart-home environment.

In one example, responsive to receiving a message having informationrelated to a condition of the smart-home environment, at least one ofthe smart nodes is configured to: determine an objective to beaccomplished in the smart-home environment, the objective beingappropriate in light of the condition; determine a function to beperformed in the smart-home environment for the purpose of accomplishingthe objective; and transmit to the other low-powered and smart nodes ofthe smart-home environment instructions to perform the function in thesmart-home environment for the purpose of accomplishing the objective.In another example, responsive to receiving a message having informationrelated to a condition of the smart-home environment, at least one ofthe smart nodes is configured to transmit to a server a message havinginformation related to a condition of the smart-home environment.According to this example, responsive to receiving a message havinginformation related to the condition of the smart-home environment, theserver is configured to: determine an objective to be accomplished inthe smart-home environment, the objective being appropriate in light ofthe condition; determine a function to be performed in the smart-homeenvironment for the purpose of accomplishing the objective; and transmitto at least one of the smart nodes of the smart-home environmentinstructions to perform the function in the smart-home environment forthe purpose of accomplishing the objective.

According to embodiments, the low-powered nodes are capable ofcommunicating using a first wireless protocol characterized byrelatively low power usage and relatively low data rates, and whereinthe smart nodes are capable of communicating using the first wirelessprotocol and a second wireless protocol characterized by relativelyhigher power usage and relatively higher data rates. According toembodiments, when transmitting to a server a message having informationrelated to a condition of the smart-home environment, at least one ofthe smart nodes serves as a communication bridge to a wide area networkusing the second wireless protocol characterized by relatively higherpower usage and relatively higher data rates.

According to embodiments, the smart nodes can smart devices. The smartdevices are, for example, one or more of a hazard detector unit, adoorbell, a thermostat, a wall switch, and a wall plug. In otherembodiments, the low-powered nodes are smart devices. In some examples,the smart devices are identical and capable of performing as thelow-powered node and the smart node. Example conditions to be monitoredinclude one or more of a temperature, an amount of light, a sound, amovement, a vibration, a smell, a toxin, and an amount of heat. Examplefunctions and corresponding objectives include one or more of triggeringan alarm to secure the smart-home environment, adjusting a thermostatsetting to make the smart-home environment comfortable, and turning onor off a light to secure the smart-home environment or for use byoccupants of the smart-home environment.

According to another embodiment, a method is provided, comprising:monitoring, by one or more communicatively interconnected low-poweredand smart nodes, a condition of a smart-home environment; transmitting,by at least one of the low-powered and smart nodes, a message havinginformation related to the condition of the smart-home environment;receiving, by at least one of the smart nodes, a message havinginformation related to the condition in the smart-home environment;determining, by at least one of the smart nodes based at least in parton the information related to the condition, a function to be performedin the smart-home environment; transmitting, by at least one of thesmart nodes, a message that causes at least one of the low-powered andsmart nodes to perform the function in the smart-home environment. Insome embodiments, transmitting, by at least one of the low-powered andsmart nodes, a message having information related to the condition ofthe smart-home environment involves “repeating” the message to the otherlow-powered and smart nodes in the smart-home environment.

According to another embodiment, a home automation system is provided,comprising: a home security system including one or more sensor nodescapable of detecting motion within a home; and a service robot systemincluding at least one service robot that autonomously moves within thehome to perform one or more functions; wherein the home security systemand the service robot system are mutually configured such that therewill not be a motion alarm set off by the robot when the robot moveswithin a range of the one or more sensor nodes. According to embodimentsthe home automation system further comprises a computing device providedin operative communication with the home security system and the servicerobot system, the computing device configured to: receive in-homelocation coordinates from the robot; and distinguish between activityassociated with the robot and unexpected intrusion activity by using thein-home location coordinates to filter signals from the one more sensornodes. In some examples, the computing device is an on-site computingdevice. In other examples, the computing device is a remote server.According to embodiments, the robot outputs a signal when moving toalert the home security system to its presence so that there will not bea motion alarm set off for as long as the signal is detected by the oneor more sensor nodes.

In some embodiments of the home automation system, the signal isauthenticated and encrypted such that the signal cannot be learned andreplicated by a potential burglar. In some examples, signalauthentication and encryption is accomplished by a permissionrequest-response scheme, wherein the service robot system requestspermission from home security system when the service robot system isready to deploy the robot to perform the one or more function. Thesignal can be, for example, one or more of an optical notifying signal,an audible notifying signal, an infrared notifying signal, an infrasonicnotifying signal, and a wirelessly transmitted data notification signal.According to embodiments, the service robot system does not deploy therobot until receiving a “yes” message from home security system. Examplefunctions performed by the robot include, but are not limited to, one ormore of floor sweeping, floor washing, playing music for an occupant,serving as a localized thermostat for an occupant, serving as alocalized air monitor/purifier for an occupant, serving as a localizedbaby monitor, and serving as a localized hazard detector for anoccupant.

According to another embodiment, a wall switch controller is provided,comprising: a docking station configured to receive a user-removablewall-switch head unit, the docking station configured to be permanentlyconnected to a wall and coupled to high-power voltage wires; and auser-removable wall-switch head unit configured to be user-insertableinto said docking station and user-removable therefrom such that theuser is not exposed to high-voltage connections when inserting orremoving. In some embodiments, the wall switch controller furthercomprises inputs and outputs and circuitry for switchably controllinghousehold line current power to at least one household electricalfixture; one or more of an occupancy sensor, a temperature sensor, and aprocessor; circuitry for leveraging electrical power from the householdline current power inputs to power one or more of the occupancy sensor,the temperature sensor, and the processor; wireless communicationcircuitry configured to communicate with a plurality of low-powereddevices using a first wireless protocol characterized by relatively lowpower usage and relatively lower data rates and to serve as acommunications bridge to a wide area network using a second wirelessprotocol characterized by relatively higher power usage and relativelyhigher data rates; a power storage module for storing power and usingthe power to maintain wireless communications during a power outage; ahousing configured to be compatible for installation in a conventionalsingle-gang or multiple-gang wall box.

According to another embodiment, a method is provided for providing analarm clock in a smart-home environment, comprising: obtaining, at aserver, a wake time for an occupant of the smart-home environment; andinstructing, by the server, one or more smart devices in the smart-homeenvironment to output an audible alarm when the wake time occurs.According to embodiments, obtaining a wake time comprises: obtainingover a period, from one or more motion-sensing smart devices located inthe smart-home environment, data related to movement of the occupant inresponse to an alarm; and inferring, by the server, a wake time based atleast in part on the data related to movement of the occupant inresponse to an alarm over the period. According to embodiments, themethod further comprises: tracking, by the one or more motion-detectingsmart devices, movement of the occupant between one or more locationswithin the smart-home environment; detecting when the occupant stopsmoving for a period; inferring that the occupant has fallen asleep at alocation within the smart-home environment; inferring where and when theoccupant prefers to sleep.

According to embodiments, instructing one or more smart devices in thesmart-home environment to output an audible alarm when the wake timeoccurs, comprises: instructing only the one of the one or more smartdevices that is closest to the occupant to output the audible alarm. Insome examples, tracking a location of the occupant within the smart-homeenvironment, comprises: generating a unique signature for the occupantbased at least in part of data obtained from the one or more smartdevices; and using the unique signature to distinguish the occupant fromother occupants of the smart-home environment. The unique signature ofthe occupant, according to some embodiments, is based on one or more ofa walking gate, patterns of movement, voice, height, and size. The datathat is obtained from the one or more smart devices and used to generatethe unique signature is, for example, obtained from at least one of anultrasonic sensor and a passive IR sensor included in the one or moremotion-detecting smart devices.

According to embodiments, the method of providing an alarm clock furthercomprising: transmitting, to a thermostat, instructions to heat or coolthe smart-home environment to a desired “sleeping” temperature settingwhen the occupant is determined to be sleeping; and transmitting, to athermostat, instructions to heat or cool the smart-home environment to adesired “awake” temperature setting when the occupant is not determinedto be sleeping. In some examples, the desired “sleeping” and “awake”temperature settings can be learned over time, such as be observingwhich temperature the occupant sets the thermostat to before going tosleep and which temperature the occupant sets the thermostat to uponwaking up. According to embodiments, the method further comprisestransmitting, to a thermostat, instructions to pre-heat or pre-cool thesmart-home environment to a desired “awake” temperature setting inanticipation of the occupant waking up.

According to another embodiment, a method is provide, the methodcomprises: obtaining, at a server from one or more smart devices locatedin one or more smart-home environments, home data collected from thesmart-home environments by one or more sensors of the smart devices; andexposing, by the server, one or more application program interfaces foraccess by one or more service-provider entities to obtain the home datafor the purpose of optimizing one or more home services provided by theone or more service-provider entities to the one or more smart-homeenvironments. The data, for example, includes at least one of smartdevice usage patterns and occupant presence patterns. In someembodiments, the server derives aggregate patterns from the home data.According to embodiments, the server provides neighborhood-levelprobabilities regarding whether occupants may be currently present inthe smart-home environments. According to embodiments, the one or morehome services being optimized are delivery services whose effectivenessis improved when the occupants are at home. According to embodiments,the application program interfaces anonymize the home data such that anyone of the smart-home environments is not identifiable. According toembodiments, the one or more home services being optimized are notfunctionally related to the home data being collected by the one or moresmart sensors.

According to another embodiment, a home monitoring and control system isprovided. The system comprises: a plurality of hazard detectors withwireless capability and occupancy detectors and audio speakers; and adoorbell having a processor and wireless capability; wherein adoorbell-ringing event triggers the processor to cause the doorbell towirelessly communicate with one or more of the hazard detectors to causethe audio speakers to alert occupants of a visitor at the door. In someembodiments, each hazard detector can be configured with ado-not-disturb setting such that it will remain silent under one or morepre-determined conditions. According to embodiments, the pre-determinedconditions include one or more of an occupant determined to be sleepingproximate to the hazard detector, room location of hazard detector isunoccupied, hazard detector is located in an occupied bedroom; andhazard detector is located in a kid's bedroom.

For a more complete understanding of the nature and advantages ofembodiments of the present invention, reference should be made to theensuing detailed description and accompanying drawings. Other aspects,objects and advantages of the invention will be apparent from thedrawings and detailed description that follows. However, the scope ofthe invention will be fully apparent from the recitations of the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 an example of a smart-home environment within which one or moreof the devices, methods, systems, services, and/or computer programproducts described further herein will be applicable, according toembodiments.

FIG. 2 illustrates a network-level view of an extensible devices andservices platform with which the smart-home environment of FIG. 1 can beintegrated, according to embodiments.

FIG. 3 illustrates an abstracted functional view of the extensibledevices and services platform of FIG. 2, with reference to a processingengine as well as devices of the smart-home environment, according toembodiments.

FIG. 4A is a simplified block diagram illustrating components of a wallswitch according to embodiments.

FIGS. 4B-C illustrate example modular head units for the wall switch ofFIG. 4A, according to embodiments.

FIG. 5 is a simplified block diagram illustrating components of anintelligent, multi-sensing, network-connected wall light switch,according to embodiments.

FIG. 6 is a schematic diagram illustrating a silence gesture forremotely deactivating an alarm, according to embodiments.

FIG. 7 is a schematic diagram illustrating a silence gesture forremotely deactivating an alarm, according to embodiments.

FIGS. 8A-B are simplified block diagrams illustrating components of anintelligent, multi-sensing, network-connected entryway interface device,according to embodiments.

FIG. 9 is a schematic diagram illustrating an intelligent,multi-sensing, network-connected wall plug, according to embodiments.

FIGS. 10A-C are schematic diagrams illustrating an intelligent,multi-sensing, network-connected thermostat, according to embodiments.

FIG. 11 illustrates a block diagram of embodiments of a computer system.

FIG. 12 illustrates a block diagram of embodiments of a special-purposecomputer.

FIG. 13 is a schematic diagram illustrating example geo-fencing,according to embodiments.

DETAILED DESCRIPTION

Embodiments of the present invention generally relate to a plurality ofdevices, including intelligent, multi-sensing, network-connecteddevices, that communicate with each other and/or with a central serveror a cloud-computing system to provide any of a variety of useful smarthome objectives.

Various aspects and possible implementations of providing smart-homeobjectives are disclosed herein. Turning to the figures, FIG. 1illustrates an example of a smart-home environment 100 within which oneor more of the devices, methods, systems, services, and/or computerprogram products described further herein can be applicable. Thedepicted smart-home environment 100 includes a structure 150, which caninclude, e.g., a house, office building, garage, or mobile home. Itshould be appreciated that the smart-home environment 100 includes areasoutside the home, such as curtilage, the yard, and other nearby land. Itwill be appreciated that devices can also be integrated into asmart-home environment 100 that does not include an entire structure150, such as an apartment, condominium, or office space. Further, thesmart home environment can control and/or be coupled to devices outsideof the actual structure 150. Indeed, several devices in the smart homeenvironment need not physically be within the structure 150 at all. Forexample, a device controlling a pool heater or irrigation system can belocated outside of the structure 150.

The depicted structure 150 includes a plurality of rooms 152, separatedat least partly from each other via walls 154. The walls 154 can includeinterior walls or exterior walls. Each room can further include a floor156 and a ceiling 158. Devices can be mounted on, integrated with and/orsupported by a wall 154, floor 156 or ceiling 158.

In some embodiments, the smart-home environment 100 of FIG. 1 includes aplurality of devices, including intelligent, multi-sensing,network-connected devices, that can integrate seamlessly with each otherand/or with a central server or a cloud-computing system to provide anyof a variety of useful smart-home objectives. The smart-home environment100 may include one or more intelligent, multi-sensing,network-connected thermostats 102 (herein after referred to as “smartthermostats 102”), one or more intelligent, network-connected,multi-sensing hazard detection units 104 (herein after referred to as“smart hazard detectors 104”), and one or more intelligent,multi-sensing, network-connected entryway interface devices 106 (hereinafter referred to as “smart doorbells 104”). According to embodiments,the smart thermostat 102 detects ambient climate characteristics (e.g.,temperature and/or humidity) and controls a HVAC system 103 accordingly.The smart hazard detector 104 may detect the presence of a hazardoussubstance or a substance indicative of a hazardous substance (e.g.,smoke, fire, or carbon monoxide). The smart doorbell 106 may detect aperson's approach to or departure from a location (e.g., an outer door),control doorbell functionality, announce a person's approach ordeparture via audio or visual means, or control settings on a securitysystem (e.g., to activate or deactivate the security system whenoccupant go and come).

In some embodiments, the smart-home environment 100 of FIG. 1 furtherincludes one or more intelligent, multi-sensing, network-connected wallswitches 108 (herein after referred to as “smart wall switches 108”),along with one or more intelligent, multi-sensing, network-connectedwall plug interfaces 110 (herein after referred to as “smart wall plugs110”). The smart wall switches 108 may detect ambient lightingconditions, detect room-occupancy states, and control a power and/or dimstate of one or more lights. In some instances, smart wall switches 108may also control a power state or speed of a fan, such as a ceiling fan.The smart wall plugs 110 may detect occupancy of a room or enclosure andcontrol supply of power to one or more wall plugs (e.g., such that poweris not supplied to the plug if nobody is at home).

Still further, in some embodiments, the smart-home environment 100 ofFIG. 1 includes a plurality of intelligent, multi-sensing,network-connected appliances 112 (herein after referred to as “smartappliances 112”), such as refrigerators, stoves and/or ovens,televisions, washers, dryers, lights, stereos, intercom systems,garage-door openers, floor fans, ceiling fans, wall air conditioners,pool heaters, irrigation systems, security systems, and so forth.According to embodiments, the network-connected appliances 112 are madecompatible with the smart-home environment by cooperating with therespective manufacturers of the appliances. For example, the appliancescan be space heaters, window AC unites, motorized duct vents, etc. Whenplugged in, an appliance can announce itself to the smart-home network,such as by indicating what type of appliance it is, and it canautomatically integrate with the controls of the smart-home. Suchcommunication by the appliance to the smart home can be facilitated anywired or wireless communication protocols known by those having ordinaryskill in the art. The smart home also can include a variety ofnon-communicating legacy appliances 140, such as old conventionalwasher/dryers, refrigerators, and the like which can be controlled,albeit coarsely (ON/OFF), by virtue of the smart wall plugs 110. Thesmart-home environment 100 can further include a variety of partiallycommunicating legacy appliances 142, such as infrared (“IR”) controlledwall air conditioners or other IR-controlled devices, which can becontrolled by IR signals provided by the smart hazard detectors 104 orthe smart wall switches 108.

According to embodiments, the smart thermostats 102, the smart hazarddetectors 104, the smart doorbells 106, the smart wall switches 108, thesmart wall plugs 110, and other devices of the smart-home environment100 are modular and can be incorporated into older and new houses. Forexample, the devices are designed around a modular platform consistingof two basic components: a head unit and a backplate, which is alsoreferred to as a docking station. Multiple configurations of the dockingstation are provided so as to be compatible with any home, such as olderand newer homes. However, all of the docking stations include a standardhead-connection arrangement, such that any head unit can be removablyattached to any docking station. Thus, in some embodiments, the dockingstations are interfaces that serve as physical connections to thestructure and the voltage wiring of the homes, and the interchangeablehead units contain all of the sensors, processors, user interfaces, thebatteries, and other functional components of the devices.

Many different commercial and functional possibilities for provisioning,maintenance, and upgrade are possible. For example, after years of usingany particular head unit, a user will be able to buy a new version ofthe head unit and simply plug it into the old docking station. There arealso many different versions for the head units, such as low-costversions with few features, and then a progression ofincreasingly-capable versions, up to and including extremely fancy headunits with a large number of features. Thus, it should be appreciatedthat the various versions of the head units can all be interchangeable,with any of them working when placed into any docking station. This canadvantageously encourage sharing and re-deployment of old head units—forexample, when an important high-capability head unit, such as a hazarddetector, is replaced by a new version of the head unit, then the oldhead unit can be re-deployed to a backroom or basement, etc. Accordingto embodiments, when first plugged into a docking station, the head unitcan ask the user (by 2D LCD display, 2D/3D holographic projection, voiceinteraction, etc.) a few simple questions such as, “Where am I” and theuser can indicate “living room”, “kitchen” and so forth.

The smart-home environment 100 may also include communication withdevices outside of the physical home but within a proximate geographicalrange of the home. For example, the smart-home environment 100 mayinclude a pool heater monitor 114 that communicates a current pooltemperature to other devices within the smart-home environment 100 orreceives commands for controlling the pool temperature. Similarly, thesmart-home environment 100 may include an irrigation monitor 116 thatcommunicates information regarding irrigation systems within thesmart-home environment 100 and/or receives control information forcontrolling such irrigation systems. According to embodiments, analgorithm is provided for considering the geographic location of thesmart-home environment 100, such as based on the zip code or geographiccoordinates of the home. The geographic information is then used toobtain data helpful for determining optimal times for watering, suchdata may include sun location information, temperature, due point, soiltype of the land on which the home is located, etc.

By virtue of network connectivity, one or more of the smart-home devicesof FIG. 1 can further allow a user to interact with the device even ifthe user is not proximate to the device. For example, a user cancommunicate with a device using a computer (e.g., a desktop computer,laptop computer, or tablet) or other portable electronic device (e.g., asmartphone) 166. A webpage or app can be configured to receivecommunications from the user and control the device based on thecommunications and/or to present information about the device'soperation to the user. For example, the user can view a current setpointtemperature for a device and adjust it using a computer. The user can bein the structure during this remote communication or outside thestructure.

As discussed, users can control the smart thermostat and other smartdevices in the smart-home environment 100 using a network-connectedcomputer or portable electronic device 166. In some examples, some orall of the occupants (e.g., individuals who live in the home) canregister their device 166 with the smart-home environment 100. Suchregistration can be made at a central server to authenticate theoccupant and/or the device as being associated with the home, and togive permission to the occupant to use the device to control the smartdevices in the home. An occupant can use their registered device 166 toremotely control the smart devices of the home, such as when theoccupant is at work or on vacation. The occupant may also use theirregistered device to control the smart devices when the occupant isactually located inside the home, such as when the occupant sitting on acouch inside the home. It should be appreciated that instead of or inaddition to registering devices 166, the smart-home environment 100makes inferences about which individuals live in the home and aretherefore occupants and which devices 166 are associated with thoseindividuals. As such, the smart-home environment “learns” who is anoccupant and permits the devices 166 associated with those individualsto control the smart devices of the home.

In some instances, guests desire to control the smart devices. Forexample, the smart-home environment may receive communication from anunregistered mobile device of an individual inside of the home, wheresaid individual is not recognized as an occupant of the home. Further,for example, smart-home environment may receive communication from amobile device of an individual who is known to be or who is registeredas a guest.

According to embodiments, a guest-layer of controls can be provided toguests of the smart-home environment 100. The guest-layer of controlsgives guests access to basic controls (e.g., a judicially selectedsubset of features of the smart devices), such as temperatureadjustments, but it locks out other functionalities. The guest layer ofcontrols can be thought of as a “safe sandbox” in which guests havelimited controls, but they do not have access to more advanced controlsthat could fundamentally alter, undermine, damage, or otherwise impairthe occupant-desired operation of the smart devices. For example, theguest layer of controls won't permit the guest to adjust the heat-pumplockout temperature.

A use case example of this is when a guest in a smart home, the guestcould walk up to the thermostat and turn the dial manually, but theguest may not want to walk the house “hunting” the thermostat,especially at night while the home is dark and others are sleeping.Further, the guest may not want to go through the hassle of downloadingthe necessary application to their device for remotely controlling thethermostat. In fact, the guest may not have to the home owner's logincredentials, etc., and therefore cannot remotely control the thermostatvia such an application. Accordingly, according to embodiments of theinvention, the guest can open a mobile browser on their mobile device,type a keyword, such as “NEST” into the URL field and tap “Go” or“Search”, etc. In response the device presents with guest with a userinterface, such as Thermozilla UI, which allows the guest to move thetarget temperature between a limited range, such as 65 and 80 degreesFahrenheit. As discussed, the user interface provides a guest layer ofcontrols that are limited to basic functions. The guest cannot changethe target humidity, modes, or view energy history.

According to embodiments, to enable guests to access the user interfacethat provides the guest layer of controls, a local webserver is providedthat is accessible in the local area network (LAN). It does not requirea password, because physical presence inside the home is establishedreliably enough by the guest's presence on the LAN. In some embodiments,during installation of the smart device, such as the smart thermostat,the home owner is asked if they want to enable a Local Web App (LWA) onthe smart device. Business owners will likely say no; home owners willlikely say yes. When the LWA option is selected, the smart devicebroadcasts to the LAN that the above referenced keyword, such as “NEST”,is now a host alias for its local web server. Thus, no matter whose homea guest goes to, that same keyword (e.g., “NEST” is always the URL youuse to access the LWA, provided the smart device is purchased from thesame manufacturer. Further, according to embodiments, if there is morethan one smart device on the LAN, the second and subsequent smartdevices do not offer to set up another LWA. Instead, they registerthemselves as target candidates with the master LWA. And in this casethe LWA user would be asked which smart device they want to change thetemperature on before getting the simplified user interface, such asThermozilla UI, for the particular smart device they choose.

According to embodiments, a guest layer of controls may also be providedto users by means other than a device 166. For example, the smartdevice, such as the smart thermostat, may be equipped withwalkup-identification technology (e.g., face recognition, RFID,ultrasonic sensors) that “fingerprints” or creates a “signature” for theoccupants of the home. The walkup-identification technology can be thesame as or similar to the fingerprinting and signature creatingtechniques descripted in other sections of this application. Inoperation, when a person who does not live in the home or is otherwisenot registered with or whose fingerprint or signature is not recognizedby the smart home “walks up” to a smart device, the smart devicesprovides the guest with the guest layer of controls, rather than fullcontrols.

As described below, the smart thermostat and other smart devices “learn”by observing occupant behavior. For example, the smart thermostat learnsoccupants preferred temperature set-points for mornings and evenings,and it learns when the occupants are asleep or awake, as well as whenthe occupants are typically away or at home, for example. According toembodiments, when a guest controls the smart devices, such as the smartthermostat, the smart devices do not “learn” from the guest. Thisprevents the guest's adjustments and controls from affecting the learnedpreferences of the occupants.

According to some embodiments, a smart television remote control isprovided. The smart remote control recognizes occupants by thumbprint,visual identification, RFID, etc., and it recognizes users as guests oras someone belonging to a particular class having limited control andaccess (e.g., child). Upon recognizing the user as a guest or someonebelonging to a limited class, the smart remote control only permits thatuser to view a subset of channels and to make limited adjustments to thesettings of the television and other devices. For example, a guestcannot adjust the digital video recorder (DVR) settings, and a child islimited to viewing child-appropriate programming.

According to some embodiments, similar controls are provided for otherinstruments, utilities, and devices in the house. For example, sinks,bathtubs, and showers can be controlled by smart spigots that recognizeusers as guests or as children and therefore prevents water fromexceeding a designated temperature that is considered safe.

In some embodiments, in addition to containing processing and sensingcapabilities, each of the devices 102, 104, 106, 108, 110, 112, 114, and116 (collectively referred to as “the smart devices”) is capable of datacommunications and information sharing with any other of the smartdevices, as well as to any central server or cloud-computing system orany other device that is network-connected anywhere in the world. Therequired data communications can be carried out using any of a varietyof custom or standard wireless protocols (Wi-Fi, ZigBee, 6LoWPAN, etc.)and/or any of a variety of custom or standard wired protocols (CAT6Ethernet, HomePlug, etc.)

According to embodiments, all or some of the smart devices can serve aswireless or wired repeaters. For example, a first one of the smartdevices can communicate with a second one of the smart device via awireless router 160. The smart devices can further communicate with eachother via a connection to a network, such as the Internet 162. Throughthe Internet 162, the smart devices can communicate with a centralserver or a cloud-computing system 164. The central server orcloud-computing system 164 can be associated with a manufacturer,support entity, or service provider associated with the device. For oneembodiment, a user may be able to contact customer support using adevice itself rather than needing to use other communication means suchas a telephone or Internet-connected computer. Further, software updatescan be automatically sent from the central server or cloud-computingsystem 164 to devices (e.g., when available, when purchased, or atroutine intervals).

According to embodiments, the smart devices combine to create a network,such as a mesh network, of spokesman and low-power nodes in thesmart-home environment 100, where some of the smart devices are“spokesman” nodes and others are “low-powered” nodes. Spokesman nodesare sometimes referred to herein as “smart” nodes. It should beappreciated that non-smart devices may perform as lower-powered nodes.The spokesman and low-powered nodes are communicatively interconnectedand operate to accomplish a common objective or to achieve a common goalin the smart-home environment. In some embodiments, some or all of thespokesman and low-powered nodes perform one or more functions in acoordinate manner to accomplish the common objection. Example functionsand objectives include, but are not limited to, triggering an alarm forthe objective securing the home, adjusting a thermostat setting for theobjective of making the home comfortable, and turning on and of lightsfor the objective of securing the home or for use by occupants. Otherexample objective and functions are provided throughout this document.Some of the smart devices in the smart-home environment 100 are batterypowered, while others have a regular and reliable power source, such asby connecting to wiring (e.g., to 120V line voltage wires) behind thewalls 154 of the smart-home environment. The smart devices that have aregular and reliable power source are referred to as “spokesman” or“smart” nodes. These nodes are equipped with the capability of using anywireless protocol or manner to facilitate bidirectional communicationwith any of a variety of other devices in the smart-home environment 100as well as with the central server or cloud-computing system 164. On theother hand, the devices that are battery powered are referred to as“low-power” nodes. These nodes tend to be smaller than spokesman nodesand can only communicate using wireless protocol that requires verylittle power, such as Zigbee, 6LoWPAN, etc. Further, some, but not all,low-power nodes are incapable of bidirectional communication. Theselow-power nodes send messages, but they are unable to “listen”. Thus,other devices in the smart-home environment 100, such as the spokesmannodes, cannot send information to these low-power nodes.

As described, the smart devices serve as low-power and spokesman nodesto create a mesh network in the smart-home environment 100. Individuallow-power nodes in the smart-home environment regularly send outmessages regarding what they are sensing, and the other lower-powerednodes in the smart-home environment—in addition to sending out their ownmessages—repeat the messages, thereby causing the messages to travelfrom node to node (i.e., device to device) throughout the smart-homeenvironment 100. The spokesman nodes in the smart-home environment 100are able to “drop down” to low-powered communication protocols toreceive these messages, translate the messages to other communicationprotocols, and send the translated messages to other spokesman nodesand/or the central server or cloud-computing system 164. Thus, thelower-powered nodes using low-power communication protocols are ablesend messages across the entire smart-home environment 100 as well asover the Internet 162 to the central server or cloud-computing system164. According to embodiments, the mesh network enables the centralserver or cloud-computing system 164 regularly receive data from all ofthe smart devices in the home, make inferences based on the data, andsend commands back to individual one of the smart devices to accomplishsome of the smart-home objectives descried herein.

As described, the spokesman nodes and some of the lower-powered nodesare capable of “listening”. Accordingly, users, other devices, and thecentral server or cloud-computing system 164 can communicate controls tothe lower-powered nodes. For example, a user can use the portableelectronic device (e.g., a smartphone) 166 to send commands over theInternet to the central server or cloud-computing system 164, which thenrelays the commands to the spokesman nodes in the smart-home environment100. The spokesman nodes drop down to a low-power protocol tocommunicate the commands to the low-power nodes throughout thesmart-home environment, as well as to other spokesman nodes that did notreceive the commands directly from the central server or cloud-computingsystem 164. In some embodiments, the low-powered nodes and the spokesmannodes are the same type of device (e.g., hazard detector, thermostat,wall plug, etc.). In some embodiments, the low-powered and spokesmannodes are identical. For example, in some embodiments, all of thelow-powered and spokesman nodes have the same stock-keeping unit (SKU)and/or are capable of performing any role, such as performing the roleof low-powered and/or spokesman node.

An example of a low-power node is a smart nightlight 170. In addition tohousing a light source, the smart nightlight 170 houses an occupancysensor, such as an ultrasonic or passive IR sensor, and an ambient lightsensor, such as a photoresistor or a single-pixel sensor that measureslight in the room. In some embodiments, the smart nightlight 170 isconfigured to activate the light source when its ambient light sensordetects that the room is dark and when its occupancy sensor detects thatsomeone is in the room. In other embodiments, the smart nightlight 170is simply configured to activate the light source when its ambient lightsensor detects that the room is dark. Further, according to embodiments,the smart nightlight 170 includes a low-power wireless communicationchip (e.g., ZigBee chip) that regularly sends out messages regarding theoccupancy of the room and the amount of light in the room, includinginstantaneous messages coincident with the occupancy sensor detectingthe presence of a person in the room. As mentioned above, these messagesmay be sent wirelessly, using the mesh network, from node to node (i.e.,smart device to smart device) within the smart-home environment 100 aswell as over the Internet 162 to the central server or cloud-computingsystem 164.

Other examples of lower-powered nodes include battery-operated versionsof the smart hazard detectors 104. These smart hazard detectors 104 areoften located in an area without access to constant and reliable powerand, as discussed in detail below, may include any number and type ofsensors, such as smoke/fire/heat sensors, carbon monoxide/dioxidesensors, occupancy/motion sensors, ambient light sensors, temperaturesensors, humidity sensors, and the like. Furthermore, smart hazarddetectors 104 can send messages that correspond to each of therespective sensors to the other devices and the central server orcloud-computing system 164, such as by using the mesh network asdescribed above.

Examples of spokesman nodes include smart doorbells 106, smartthermostats 102, smart wall switches 108, and smart wall plugs 110.These devices 102, 106, 108, and 110 are often located near andconnected to a reliable power source, and therefore can include morepower-consuming components, such as one or more communication chipscapable of bidirectional communication in any variety of protocols.

In some embodiments, these low-powered and spokesman nodes (e.g.,devices 102, 104, 106, 108, 110, 112, and 170) can function as“tripwires” for an alarm system in the smart-home environment. Forexample, in the event a perpetrator circumvents detection by alarmsensors located at windows, doors, and other entry points of thesmart-home environment 100, the alarm could be triggered upon receivingan occupancy, motion, heat, sound, etc. message from one or more of thelow-powered and spokesman nodes in the mesh network. For example, uponreceiving a message from a smart nightlight 170 indicating the presenceof a person, the central server or cloud-computing system 164 or someother device could trigger an alarm, provided the alarm is arm at thetime of detection. Thus, the alarm system could be enhanced by variouslow-powered and spokesman nodes located throughout the smart-homeenvironment 100. In this example, a user could enhance the security ofthe smart-home environment 100 by buying and installing extra smartnightlights 170.

In some embodiments, the mesh network can be used to automatically turnon and off lights as a person transitions from room to room. Forexample, the low-powered and spokesman nodes (e.g., devices 102, 104,106, 108, 110, 112, and 170) detect the person's movement through thesmart-home environment and communicate corresponding messages throughthe mesh network. Using the messages that indicate which rooms areoccupied, the central server or cloud-computing system 164 or some otherdevice activates and deactivates the smart wall switches 108 toautomatically provide light as the person moves from room to room in thesmart-home environment 100. Further, users may provide pre-configurationinformation that indicates which smart wall plugs 110 provide power tolamps and other light sources, such as the smart nightlight 170.Alternatively, this mapping of light sources to wall plugs 110 can bedone automatically (e.g., the smart wall plugs 110 detect when a lightsource is plugged into it, and it sends a corresponding message to thecentral server or cloud-computing system 164). Using this mappinginformation in combination with messages that indicate which rooms areoccupied, the central server or cloud-computing system 164 or some otherdevice activates and deactivates the smart wall plugs 110 that providepower to lamps and other light sources so as to track the person'smovement and provide light as the person moves from room to room.

In some embodiments, the mesh network of low-powered and spokesman nodescan be used to provide exit lighting in the event of an emergency. Insome instances, to facilitate this, users provide pre-configurationinformation that indicates exit routes in the smart-home environment100. For example, for each room in the house, the user provides a map ofthe best exit route. It should be appreciated that instead of a userproviding this information, the central server or cloud-computing system164 or some other device could the automatically determine the routesusing uploaded maps, diagrams, architectural drawings of the smart-homehouse, as well as using a map generated based on positional informationobtained from the nodes of the mesh network (e.g., positionalinformation from the devices is used to construct a map of the house).In operation, when an alarm is activated (e.g., when one or more of thesmart hazard detector 104 detects smoke and activates an alarm), thecentral server or cloud-computing system 164 or some other device usesoccupancy information obtained from the low-powered and spokesman nodesto determine which rooms are occupied and then turns on lights (e.g.,nightlights 170, wall switches 108, wall plugs 110 that power lamps,etc.) along the exit routes from the occupied rooms so as to provideemergency exit lighting.

Further included and illustrated in the exemplary smart-home environment100 of FIG. 1 are service robots 162 each configured to carry out, in anautonomous manner, any of a variety of household tasks. For someembodiments, the service robots 162 can be respectively configured toperform floor sweeping, floor washing, etc. in a manner similar to thatof known commercially available devices such as the ROOMBA™ and SCOOBA™products sold by iRobot, Inc. of Bedford, Mass. Tasks such as floorsweeping and floor washing can be considered as “away” or “while-away”tasks for purposes of the instant description, as it is generally moredesirable for these tasks to be performed when the occupants are notpresent. For other embodiments, one or more of the service robots 162are configured to perform tasks such as playing music for an occupant,serving as a localized thermostat for an occupant, serving as alocalized air monitor/purifier for an occupant, serving as a localizedbaby monitor, serving as a localized hazard detector for an occupant,and so forth, it being generally more desirable for such tasks to becarried out in the immediate presence of the human occupant. Forpurposes of the instant description, such tasks can be considered as“human-facing” or “human-centric” tasks.

When serving as a localized thermostat for an occupant, a particular oneof the service robots 162 can be considered to be facilitating what canbe called a “personal comfort-area network” for the occupant, with theobjective being to keep the occupant's immediate space at a comfortabletemperature wherever that occupant may be located in the home. This canbe contrasted with conventional wall-mounted room thermostats, whichhave the more attenuated objective of keeping a statically-definedstructural space at a comfortable temperature. According to oneembodiment, the localized-thermostat service robot 162 is configured tomove itself into the immediate presence (e.g., within five feet) of aparticular occupant who has settled into a particular location in thehome (e.g. in the dining room to eat their breakfast and read the news).The localized-thermostat service robot 162 includes a temperaturesensor, a processor, and wireless communication components configuredsuch that control communications with the HVAC system, either directlyor through a wall-mounted wirelessly communicating thermostat coupled tothe HVAC system, are maintained and such that the temperature in theimmediate vicinity of the occupant is maintained at their desired level.If the occupant then moves and settles into another location (e.g. tothe living room couch to watch television), the localized-thermostatservice robot 162 proceeds to move and park itself next to the couch andkeep that particular immediate space at a comfortable temperature.

Technologies by which the localized-thermostat service robot 162 (and/orthe larger smart-home system of FIG. 1) can identify and locate theoccupant whose personal-area space is to be kept at a comfortabletemperature can include, but are not limited to, RFID sensing (e.g.,person having an RFID bracelet, RFID necklace, or RFID key fob),synthetic vision techniques (e.g., video cameras and face recognitionprocessors), audio techniques (e.g., voice, sound pattern, vibrationpattern recognition), ultrasound sensing/imaging techniques, andinfrared or near-field communication (NFC) techniques (e.g., personwearing an infrared or NFC-capable smartphone), along with rules-basedinference engines or artificial intelligence techniques that draw usefulconclusions from the sensed information (e.g., if there is only a singleoccupant present in the home, then that is the person whose immediatespace should be kept at a comfortable temperature, and the selection ofthe desired comfortable temperature should correspond to that occupant'sparticular stored profile).

When serving as a localized air monitor/purifier for an occupant, aparticular service robot 162 can be considered to be facilitating whatcan be called a “personal health-area network” for the occupant, withthe objective being to keep the air quality in the occupant's immediatespace at healthy levels. Alternatively or in conjunction therewith,other health-related functions can be provided, such as monitoring thetemperature or heart rate of the occupant (e.g., using finely remotesensors, near-field communication with on-person monitors, etc.). Whenserving as a localized hazard detector for an occupant, a particularservice robot 162 can be considered to be facilitating what can becalled a “personal safety-area network” for the occupant, with theobjective being to ensure there is no excessive carbon monoxide, smoke,fire, etc.) in the immediate space of the occupant. Methods analogous tothose described above for personal comfort-area networks in terms ofoccupant identifying and tracking are likewise applicable for personalhealth-area network and personal safety-area network embodiments.

According to some embodiments, the above-referenced facilitation ofpersonal comfort-area networks, personal health-area networks, personalsafety-area networks, and/or other such human-facing functionalities ofthe service robots 162, are further enhanced by logical integration withother smart sensors in the home according to rules-based inferencingtechniques or artificial intelligence techniques for achieving betterperformance of those human-facing functionalities and/or for achievingthose goals in energy-conserving or other resource-conserving ways.Thus, for one embodiment relating to personal health-area networks, theair monitor/purifier service robot 162 can be configured to detectwhether a household pet is moving toward the currently settled locationof the occupant (e.g., using onboard sensors and/or by datacommunications with other smart-home sensors along with rules-basedinferencing/artificial intelligence techniques), and if so, the airpurifying rate is immediately increased in preparation for the arrivalof more airborne pet dander. For another embodiment relating to personalsafety-area networks, the hazard detector service robot 162 can beadvised by other smart-home sensors that the temperature and humiditylevels are rising in the kitchen, which is nearby to the occupant'scurrent dining room location, and responsive to this advisory the hazarddetector service robot 162 will temporarily raise a hazard detectionthreshold, such as a smoke detection threshold, under an inference thatany small increases in ambient smoke levels will most likely be due tocooking activity and not due to a genuinely hazardous condition.

The above-described “human-facing” and “away” functionalities can beprovided, without limitation, by multiple distinct service robots 162having respective dedicated ones of such functionalities, by a singleservice robot 162 having an integration of two or more different ones ofsuch functionalities, and/or any combinations thereof (including theability for a single service robot 162 to have both “away” and “humanfacing” functionalities) without departing from the scope of the presentteachings. Electrical power can be provided by virtue of rechargeablebatteries or other rechargeable methods, with FIG. 1 illustrating anexemplary out-of-the-way docking station 164 to which the service robots162 will automatically dock and recharge its batteries (if needed)during periods of inactivity. Preferably, each service robot 162includes wireless communication components that facilitate datacommunications with one or more of the other wirelessly communicatingsmart-home sensors of FIG. 1 and/or with one or more other servicerobots 162 (e.g., using Wi-Fi, Zigbee, Z-Wave, 6LoWPAN, etc.), and oneor more of the smart-home devices of FIG. 1 can be in communication witha remote server over the Internet. Alternatively or in conjunctiontherewith, each service robot 162 can be configured to communicatedirectly with a remote server by virtue of cellular telephonecommunications, satellite communications, 3G/4G network datacommunications, or other direct communication method.

Provided according to some embodiments are systems and methods relatingto the integration of the service robot(s) 162 with home securitysensors and related functionalities of the smart home system. Theembodiments are particularly applicable and advantageous when appliedfor those service robots 162 that perform “away” functionalities or thatotherwise are desirable to be active when the home is unoccupied(hereinafter “away-service robots”). Included in the embodiments aremethods and systems for ensuring that home security systems, intrusiondetection systems, and/or occupancy-sensitive environmental controlsystems (for example, occupancy-sensitive automated setback thermostatsthat enter into a lower-energy-using condition when the home isunoccupied) are not erroneously triggered by the away-service robots.

Provided according to one embodiment is a home automation and securitysystem (e.g., as shown in FIG. 1) that is remotely monitored by amonitoring service by virtue of automated systems (e.g., cloud-basedservers or other central servers, hereinafter “central server”) that arein data communications with one or more network-connected elements ofthe home automation and security system. The away-service robots areconfigured to be in operative data communication with the centralserver, and are configured such that they remain in a non-away-servicestate (e.g., a dormant state at their docking station) unless permissionis granted from the central server (e.g., by virtue of an“away-service-OK” message from the central server) to commence theiraway-service activities. An away-state determination made by the system,which can be arrived at (i) exclusively by local on-premises smartdevice(s) based on occupancy sensor data, (ii) exclusively by thecentral server based on received occupancy sensor data and/or based onreceived proximity-related information such as GPS coordinates from usersmartphones or automobiles, or (iii) any combination of (i) and (ii))can then trigger the granting of away-service permission to theaway-service robots by the central server. During the course of theaway-service robot activity, during which the away-service robots maycontinuously detect and send their in-home location coordinates to thecentral server, the central server can readily filter signals from theoccupancy sensing devices to distinguish between the away-service robotactivity versus any unexpected intrusion activity, thereby avoiding afalse intrusion alarm condition while also ensuring that the home issecure. Alternatively or in conjunction therewith, the central servermay provide filtering data (such as an expected occupancy-sensingprofile triggered by the away-service robots) to the occupancy sensingnodes or associated processing nodes of the smart home, such that thefiltering is performed at the local level. Although somewhat lesssecure, it would also be within the scope of the present teachings forthe central server to temporarily disable the occupancy sensingequipment for the duration of the away-service robot activity.

According to another embodiment, functionality similar to that of thecentral server in the above example can be performed by an on-sitecomputing device such as a dedicated server computer, a “master” homeautomation console or panel, or as an adjunct function of one or more ofthe smart-home devices of FIG. 1. In such embodiment, there would be nodependency on a remote service provider to provide the “away-service-OK”permission to the away-service robots and the false-alarm-avoidancefiltering service or filter information for the sensed intrusiondetection signals.

According to other embodiments, there are provided methods and systemsfor implementing away-service robot functionality while avoiding falsehome security alarms and false occupancy-sensitive environmentalcontrols without the requirement of a single overall event orchestrator.For purposes of the simplicity in the present disclosure, the homesecurity systems and/or occupancy-sensitive environmental controls thatwould be triggered by the motion, noise, vibrations, or otherdisturbances of the away-service robot activity are referenced simply as“activity sensing systems,” and when so triggered will yield a“disturbance-detected” outcome representative of the false trigger (forexample, an alarm message to a security service, or an “arrival”determination for an automated setback thermostat that causes the hometo be heated or cooled to a more comfortable “occupied” setpointtemperature). According to one embodiment, the away-service robots areconfigured to emit a standard ultrasonic sound throughout the course oftheir away-service activity, the activity sensing systems are configuredto detect that standard ultrasonic sound, and the activity sensingsystems are further configured such that no disturbance-detected outcomewill occur for as long as that standard ultrasonic sound is detected.For other embodiments, the away-service robots are configured to emit astandard notification signal throughout the course of their away-serviceactivity, the activity sensing systems are configured to detect thatstandard notification signal, and the activity sensing systems arefurther configured such that no disturbance-detected outcome will occurfor as long as that standard notification signal is detected, whereinthe standard notification signal comprises one or more of: an opticalnotifying signal; an audible notifying signal; an infrared notifyingsignal; an infrasonic notifying signal; a wirelessly transmitted datanotification signal (e.g., an IP broadcast, multicast, or unicastnotification signal, or a notification message sent in an TCP/IP two-waycommunication session).

According to some embodiments, the notification signals sent by theaway-service robots to the activity sensing systems are authenticatedand encrypted such that the notifications cannot be learned andreplicated by a potential burglar. Any of a variety of knownencryption/authentication schemes can be used to ensure such datasecurity including, but not limited to, methods involving third partydata security services or certificate authorities. For some embodiments,a permission request-response model can be used, wherein any particularaway-service robot requests permission from each activity sensing systemin the home when it is ready to perform its away-service tasks, and doesnot initiate such activity until receiving a “yes” or “permissiongranted” message from each activity sensing system (or from a singleactivity sensing system serving as a “spokesman” for all of the activitysensing systems). One advantage of the described embodiments that do notrequire a central event orchestrator is that there can (optionally) bemore of an arms-length relationship between the supplier(s) of the homesecurity/environmental control equipment, on the one hand, and thesupplier(s) of the away-service robot(s), on the other hand, as it isonly required that there is the described standard one-way notificationprotocol or the described standard two-way request/permission protocolto be agreed upon by the respective suppliers.

According to still other embodiments, the activity sensing systems areconfigured to detect sounds, vibrations, RF emissions, or otherdetectable environmental signals or “signatures” that are intrinsicallyassociated with the away-service activity of each away-service robot,and are further configured such that no disturbance-detected outcomewill occur for as long as that particular detectable signal orenvironmental “signature” is detected. By way of example, a particularkind of vacuum-cleaning away-service robot may emit a specific sound orRF signature. For one embodiment, the away-service environmentalsignatures for each of a plurality of known away-service robots arestored in the memory of the activity sensing systems based onempirically collected data, the environmental signatures being suppliedwith the activity sensing systems and periodically updated by a remoteupdate server. For another embodiment, the activity sensing systems canbe placed into a “training mode” for the particular home in which theyare installed, wherein they “listen” and “learn” the particularenvironmental signatures of the away-service robots for that home duringthat training session, and thereafter will suppress disturbance-detectedoutcomes for intervals in which those environmental signatures areheard.

For still another embodiment, which is particularly useful when theactivity sensing system is associated with occupancy-sensitiveenvironmental control equipment rather than a home security system, theactivity sensing system is configured to automatically learn theenvironmental signatures for the away-service robots by virtue ofautomatically performing correlations over time between detectedenvironmental signatures and detected occupancy activity. By way ofexample, for one embodiment an intelligent automatednonoccupancy-triggered setback thermostat such as the Nest LearningThermostat can be configured to constantly monitor for audible and RFactivity as well as to perform infrared-based occupancy detection. Inparticular view of the fact that the environmental signature of theaway-service robot will remain relatively constant from event to event,and in view of the fact that the away-service events will likely either(a) themselves be triggered by some sort of nonoccupancy condition asmeasured by the away-service robots themselves, or (b) will occur atregular times of day, there will be patterns in the collected data bywhich the events themselves will become apparent and for which theenvironmental signatures can be readily learned. Generally speaking, forthis automatic-learning embodiment in which the environmental signaturesof the away-service robots are automatically learned without requiringuser interaction, it is more preferable that a certain number of falsetriggers be tolerable over the course of the learning process.Accordingly, this automatic-learning embodiment is more preferable forapplication in occupancy-sensitive environmental control equipment (suchas an automated setback thermostat) rather than home security systemsfor the reason that a few false occupancy determinations may cause a fewinstances of unnecessary heating or cooling, but will not otherwise haveany serious, whereas false home security alarms may have more seriousconsequences.

According to embodiments, technologies including the sensors of thesmart devices located in the mesh network of the smart-home environmentin combination with rules-based inference engines or artificialintelligence provided at the central server or cloud-computing system164 are used to provide a personal “smart alarm clock” for individualoccupants of the home. For example, user-occupants can communicate withthe central server or cloud-computing system 164 via their mobiledevices 166 to access an interface for the smart alarm clock. There,occupants can turn on their “smart alarm clock” and input a wake timefor the next day and/or for additional days. In some embodiments, theoccupant may have the option of setting a specific wake time for eachday of the week, as well as the option of setting some or all of theinputted wake times to “repeat”. Artificial intelligence will be used toconsider the occupant's response to these alarms when they go off andmake inferences about the user's preferred sleep patterns over time.

According to embodiments, the smart device in the smart-home environment100 that happens to be closest to the occupant when the occupant fallsasleep will be the devices that transmits messages regarding when theoccupant stopped moving, from which the central server orcloud-computing system 164 will make inferences about where and when theoccupant prefers to sleep. This closest smart device will as be thedevice that sounds the alarm to wake the occupant. In this manner, the“smart alarm clock” will follow the occupant throughout the house, bytracking the individual occupants based on their “unique signature”,which is determined based on data obtained from sensors located in thesmart devices. For example, the sensors include ultrasonic sensors,passive IR sensors, and the like. The unique signature is based on acombination of walking gate, patterns of movement, voice, height, size,etc. It should be appreciated that facial recognition may also be used.

In some embodiments, the wake times associated with the “smart alarmclock” are used to by the smart thermostat 102 to control the HVAC in anefficient manner so as to pre-heat or cool the house to the occupant'sdesired “sleeping” and “awake” temperature settings. The preferredsettings can be learned over time, such as be observing whichtemperature the occupant sets the thermostat to before going to sleepand which temperature the occupant sets the thermostat to upon wakingup.

In some embodiments, a device is positioned proximate to the occupant'sbed, such as on an adjacent nightstand, and collects data as theoccupant sleeps using noise sensors, motion sensors (e.g., ultrasonic,IR, and optical), etc. Data may be obtained by the other smart devicesin the room as well. Such data may include the occupant's breathingpatterns, heart rate, movement, etc. Inferences are made based on thisdata in combination with data that indicates when the occupant actuallywakes up. For example, if—on a regular basis—the occupant's heart rate,breathing, and moving all increase by 5% to 10%, twenty to thirtyminutes before the occupant wakes up each morning, then predictions canbe made regarding when the occupant is going to wake. Other devices inthe home can use these predictions to provide other smart-homeobjectives, such as adjusting the smart thermostat 102 so as to pre-heator cool the home to the occupant's desired setting before the occupantwakes up. Further, these predictions can be used to set the “smart alarmclock” for the occupant, to turn on lights, etc.

According to embodiments, technologies including the sensors of thesmart devices location through the smart-home environment in combinationwith rules-based inference engines or artificial intelligence providedat the central server or cloud-computing system 164 are used to detectoror monitor the progress of Alzheimer's Disease. For example, the uniquesignatures of the occupants are used to track the individual occupants'movement throughout the smart-home environment 100. This data can beaggregated and analyzed to identify patterns indicative of Alzheimer's.Oftentimes, individuals with Alzheimer's have distinctive patterns ofmigration in their homes. For example, a person will walk to the kitchenand stand there for a while, then to the living room and stand there fora while, and then back to the kitchen. This pattern will take aboutthirty minutes, and then the person will repeat the pattern. Accordingto embodiments, the remote servers or cloud computing architectures 164analyze the person's migration data collected by the mesh network of thesmart-home environment to identify such patterns.

FIG. 2 illustrates a network-level view of an extensible devices andservices platform 200 with which a plurality of smart-home environments,such as the smart-home environment 100 of FIG. 1, can be integrated. Theextensible devices and services platform 200 includes remote servers orcloud computing architectures 164. Each of the intelligent,network-connected devices 102, 104, 106, 108, 110, 112, 114, and 116from FIG. 1 (identified simply as “smart devices” in FIGS. 2-3 herein)can communicate with the remote servers or cloud computing architectures164. For example, a connection to the Internet 162 can be establishedeither directly (for example, using 3G/4G connectivity to a wirelesscarrier), though a hubbed network 212 (which can be scheme ranging froma simple wireless router, for example, up to and including anintelligent, dedicated whole-home control node), or through anycombination thereof.

Although in some examples provided herein, the devices and servicesplatform 200 communicates with and collects data from the smart devicesof smart-home environment 100 of FIG. 1, it should be appreciated thatthe devices and services platform 200 communicates with and collectsdata from a plurality of smart-home environments across the world. Forexample, the central server or cloud-computing system 164 can collecthome data 202 from the devices of one or more smart-home environments,where the devices can routinely transmit home data or can transmit homedata in specific instances (e.g., when a device queries the home data202). Thus, the devices and services platform 200 routinely collectsdata from homes across the world. As described, the collected home data202 includes, for example, power consumption data, occupancy data, HVACsettings and usage data, carbon monoxide levels data, carbon dioxidelevels data, volatile organic compounds levels data, sleeping scheduledata, cooking schedule data, inside and outside temperature humiditydata, television viewership data, inside and outside noise level data,etc.

The central server or cloud-computing architecture 164 can furtherprovide one or more services 204. The services 204 can include, e.g.,software updates, customer support, sensor data collection/logging,remote access, remote or distributed control, or use suggestions (e.g.,based on collected home data 202 to improve performance, reduce utilitycost, etc.). Data associated with the services 204 can be stored at thecentral server or cloud-computing system 164 and the central server orthe cloud-computing system 164 can retrieve and transmit the data at anappropriate time (e.g., at regular intervals, upon receiving requestfrom a user, etc.).

As illustrated in FIG. 2, embodiments of the extensible devices andservices platform 200 include a processing engine 206, which can beconcentrated at a single server or distributed among several differentcomputing entities without limitation. The processing engine 206 caninclude engines configured to receive data from devices of smart-homeenvironments (e.g., via the Internet or a hubbed network), to index thedata, to analyze the data and/or to generate statistics based on theanalysis or as part of the analysis. The analyzed data can be stored asderived home data 208.

Results of the analysis or statistics can thereafter be transmitted backto the device that provided home data used to derive the results, toother devices, to a server providing a webpage to a user of the device,or to other non-device entities. For example, use statistics, usestatistics relative to use of other devices, use patterns, and/orstatistics summarizing sensor readings can be generated by theprocessing engine 206 and transmitted. The results or statistics can beprovided via the Internet 162. In this manner, the processing engine 206can be configured and programmed to derive a variety of usefulinformation from the home data 202. A single server can include one ormore engines.

The derived data can be highly beneficial at a variety of differentgranularities for a variety of useful purposes, ranging from explicitprogrammed control of the devices on a per-home, per-neighborhood, orper-region basis (for example, demand-response programs for electricalutilities), to the generation of inferential abstractions that canassist on a per-home basis (for example, an inference can be drawn thatthe homeowner has left for vacation and so security detection equipmentcan be put on heightened sensitivity), to the generation of statisticsand associated inferential abstractions that can be used for governmentor charitable purposes. For example, processing engine 206 can generatestatistics about device usage across a population of devices and sendthe statistics to device users, service providers or other entities(e.g., that have requested or may have provided monetary compensationfor the statistics).

According to some embodiments, the home data 202, the derived home data208, and/or another data can be used to create “automated neighborhoodsafety networks.” For example, in the event the central server orcloud-computing architecture 164 receives data indicating that aparticular home has been broken into, is experiencing a fire, or someother type of emergency event, an alarm is sent to other smart homes inthe “neighborhood.” In some instances, the central server orcloud-computing architecture 164 automatically identifies smart homeswithin a radius of the home experiencing the emergency and sends analarm to the identified homes. In such instances, the other homes in the“neighborhood” do not have to sign up for or register to be a part of asafety network, but instead are notified of emergency based on theirproximity to the location of the emergency. This creates robust andevolving neighborhood security watch networks, such that if one person'shome is getting broken into, an alarm can be sent to nearby homes, suchas by audio announcements via the smart devices located in those homes.It should be appreciated that this can be an opt-in service and that, inaddition to or instead of the central server or cloud-computingarchitecture 164 selecting which homes to send alerts to, individualscan subscribe to participate in such networks and individuals canspecify which homes they want to receive alerts from. This can include,for example, the homes of family members who live in different cities,such that individuals can receive alerts when their loved ones in otherlocations are experiencing an emergency.

According to some embodiments, sound, vibration, and/or motion sensingcomponents of the smart devices are used to detect sound, vibration,and/or motion created by running water. Based on the detected sound,vibration, and/or motion, the central server or cloud-computingarchitecture 164 makes inferences about water usage in the home andprovides related services. For example, the central server orcloud-computing architecture 164 can run programs/algorithms thatrecognize what water sounds like and when it is running in the home.According to one embodiment, to map the various water sources of thehome, upon detecting running water, the central server orcloud-computing architecture 164 sends a message an occupant's mobiledevice asking if water is currently running or if water has beenrecently run in the home and, if so, which room and whichwater-consumption appliance (e.g., sink, shower, toilet, etc.) was thesource of the water. This enables the central server or cloud-computingarchitecture 164 to determine the “signature” or “fingerprint” of eachwater source in the home. This is sometimes referred to herein as “audiofingerprinting water usage.”

In one illustrative example, the central server or cloud-computingarchitecture 164 creates a signature for the toilet in the masterbathroom, and whenever that toilet is flushed, the central server orcloud-computing architecture 164 will know that the water usage at thattime is associated with that toilet. Thus, the central server orcloud-computing architecture 164 can track the water usage of thattoilet as well as each water-consumption application in the home. Thisinformation can be correlated to water bills or smart water meters so asto provide users with a breakdown of their water usage.

According to some embodiments, sound, vibration, and/or motion sensingcomponents of the smart devices are used to detect sound, vibration,and/or motion created by mice and other rodents as well as by termites,cockroaches, and other insects (collectively referred to as “pests”).Based on the detected sound, vibration, and/or motion, the centralserver or cloud-computing architecture 164 makes inferences aboutpest-detection in the home and provides related services. For example,the central server or cloud-computing architecture 164 can runprograms/algorithms that recognize what certain pests sound like, howthey move, and/or the vibration they create, individually and/orcollectively. According to one embodiment, the central server orcloud-computing architecture 164 can determine the “signatures” ofparticular types of pests.

For example, in the event the central server or cloud-computingarchitecture 164 detects sounds that may be associated with pests, itnotifies the occupants of such sounds and suggests hiring a pest controlcompany. If it is confirmed that pests are indeed present, the occupantsinput to the central server or cloud-computing architecture 164confirmation that its detection was correct, along with detailsregarding the identified pests, such as name, type, description,location, quantity, etc. This enables the central server orcloud-computing architecture 164 to “tune” itself for better detectionand create “signatures” or “fingerprints” for specific types of pests.For example, the central server or cloud-computing architecture 164 canuse the tuning as well as the signatures and fingerprints to detectpests in other homes, such as nearby homes that may be experiencingproblems with the same pests. Further, for example, in the event thattwo or more homes in a “neighborhood” are experiencing problems with thesame or similar types of pests, the central server or cloud-computingarchitecture 164 can make inferences that nearby homes may also havesuch problems or may be susceptible to having such problems, and it cansend warning messages to those home to help facilitate early detectionand prevention.

In some embodiments, to encourage innovation and research and toincrease products and services available to users, the devices andservices platform 200 exposes a range of application programminginterfaces (APIs) 210 to third parties, such as charities 222,governmental entities 224 (e.g., the Food and Drug Administration or theEnvironmental Protection Agency), academic institutions 226 (e.g.,university researchers), businesses 228 (e.g., providing devicewarranties or service to related equipment, targeting advertisementsbased on home data), utility companies 230, and another other thirdparties. The APIs 210 are coupled to and permit third-party systems tocommunicate with the central server or the cloud-computing system 164,including the services 204, the processing engine 206, the home data202, and the derived home data 208. For example, the APIs 210 allowapplications executed by the third parties to initiate specific dataprocessing tasks that are executed by the central server or thecloud-computing system 164, as well as to receive dynamic updates to thehome data 202 and the derived home data 208.

For example, third parties can develop programs and/or applications,such as web or mobile apps, that integrate with the central server orthe cloud-computing system 164 to provide services and information tousers. Such programs and application may be, for example, designed tohelp users reduce energy consumption, to preemptively service faultyequipment, to prepare for high service demands, to track past serviceperformance, etc., or to perform any of a variety of beneficialfunctions or tasks now known or hereinafter developed.

According to some embodiments, third-party applications make inferencesfrom the home data 202 and the derived home data 208, such inferencesmay include when are occupants home, when are they sleeping, when arethey cooking, when are they in the den watching television, when do theyshower. The answers to these questions may help third-parties benefitconsumers by providing them with interesting information, products andservices as well as with providing them with targeted advertisements.

In one example, a shipping company creates an application that makesinferences regarding when people are at home. The application uses theinferences to schedule deliveries for times when people will most likelybe at home. The application can also build delivery routes around thesescheduled times. This reduces the number of instances where the shippingcompany has to make multiple attempts to deliver packages, and itreduces the number of time consumers have to pick up their packages fromthe shipping company.

FIG. 3 illustrates an abstracted functional view of the extensibledevices and services platform 200 of FIG. 2, with particular referenceto the processing engine 206 as well as devices, such as those of thesmart-home environment 100 of FIG. 1. Even though devices situated insmart-home environments will have an endless variety of differentindividual capabilities and limitations, they can all be thought of assharing common characteristics in that each of them is a data consumer302 (DC), a data source 304 (DS), a services consumer 306 (SC), and aservices source 308 (SS). Advantageously, in addition to providing theessential control information needed for the devices to achieve theirlocal and immediate objectives, the extensible devices and servicesplatform 200 can also be configured to harness the large amount of datathat is flowing out of these devices. In addition to enhancing oroptimizing the actual operation of the devices themselves with respectto their immediate functions, the extensible devices and servicesplatform 200 can be directed to “repurposing” that data in a variety ofautomated, extensible, flexible, and/or scalable ways to achieve avariety of useful objectives. These objectives may be predefined oradaptively identified based on, e.g., usage patterns, device efficiency,and/or user input (e.g., requesting specific functionality).

For example, FIG. 3 shows processing engine 206 as including a number ofparadigms 310. Processing engine 206 can include a managed servicesparadigm 310 a that monitors and manages primary or secondary devicefunctions. The device functions can include ensuring proper operation ofa device given user inputs, estimating that (e.g., and responding to) anintruder is or is attempting to be in a dwelling, detecting a failure ofequipment coupled to the device (e.g., a light bulb having burned out),implementing or otherwise responding to energy demand response events,or alerting a user of a current or predicted future event orcharacteristic. Processing engine 206 can further include anadvertising/communication paradigm 310 b that estimates characteristics(e.g., demographic information), desires and/or products of interest ofa user based on device usage. Services, promotions, products or upgradescan then be offered or automatically provided to the user. Processingengine 206 can further include a social paradigm 310 c that usesinformation from a social network, provides information to a socialnetwork (for example, based on device usage), and/or processes dataassociated with user and/or device interactions with the social networkplatform. For example, a user's status as reported to their trustedcontacts on the social network could be updated to indicate when theyare home based on light detection, security system inactivation ordevice usage detectors. As another example, a user may be able to sharedevice-usage statistics with other users. Yet another example, a usermay share HVAC settings that result in low power bills and other usersmay download the HVAC settings to their smart thermostat 102 to reducetheir power bills.

The processing engine 206 can include achallenges/rules/compliance/rewards paradigm 310 d that informs a userof challenges, competitions, rules, compliance regulations and/orrewards and/or that uses operation data to determine whether a challengehas been met, a rule or regulation has been complied with and/or areward has been earned. The challenges, rules or regulations can relateto efforts to conserve energy, to live safely (e.g., reducing exposureto toxins or carcinogens), to conserve money and/or equipment life, toimprove health, etc. For example, one challenge may involve participatesturning down their thermostat by one degree for one week. Those thatsuccessfully complete the challenge are rewarded, such as by coupons,virtual currency, status, etc. Regarding compliance, an example involvesa rental-property owner making a rule that no renters are permitted toaccess certain owner's rooms. The devices in the room having occupancysensors could send updates to the owner when the room is accessed.

The processing engine 206 can integrate or otherwise utilize extrinsicinformation 316 from extrinsic sources to improve the functioning of oneor more processing paradigms. Extrinsic information 316 can be used tointerpret data received from a device, to determine a characteristic ofthe environment near the device (e.g., outside a structure that thedevice is enclosed in), to determine services or products available tothe user, to identify a social network or social-network information, todetermine contact information of entities (e.g., public-service entitiessuch as an emergency-response team, the police or a hospital) near thedevice, etc., to identify statistical or environmental conditions,trends or other information associated with a home or neighborhood, andso forth.

An extraordinary range and variety of benefits can be brought about by,and fit within the scope of, the described extensible devices andservices platform 200, ranging from the ordinary to the profound. Thus,in one “ordinary” example, each bedroom of the smart-home environment100 can be provided with a smart wall switch 108, a smart wall plug 110,and/or smart hazard detectors 104, all or some of which include anoccupancy sensor, wherein the occupancy sensor is also capable ofinferring (e.g., by virtue of motion detection, facial recognition,audible sound patterns, etc.) whether the occupant is asleep or awake.If a serious fire event is sensed, the remote security/monitoringservice or fire department is advised of how many occupants there are ineach bedroom, and whether those occupants are still asleep (or immobile)or whether they have properly evacuated the bedroom. While this is, ofcourse, a very advantageous capability accommodated by the describedextensible devices and services platform, there can be substantiallymore “profound” examples that can truly illustrate the potential of alarger “intelligence” that can be made available. By way of perhaps amore “profound” example, the same data bedroom occupancy data that isbeing used for fire safety can also be “repurposed” by the processingengine 206 in the context of a social paradigm of neighborhood childdevelopment and education. Thus, for example, the same bedroom occupancyand motion data discussed in the “ordinary” example can be collected andmade available for processing (properly anonymized) in which the sleeppatterns of schoolchildren in a particular ZIP code can be identifiedand tracked. Localized variations in the sleeping patterns of theschoolchildren may be identified and correlated, for example, todifferent nutrition programs in local schools.

Turning now to FIGS. 4A-C, illustrations are provided of exemplaryembodiments of the smart wall switch 108. According to embodiments, thewall switches 108 enhance the smart-home environment 100 by providing aretrofit wall light switch that, in addition to maintaining the basiccharacter and purpose of a light switch, incorporates a host of host ofsensing, interface, and communications capabilities for enhancingoccupant comfort, convenience, and safety.

By virtue of being mounting in place of traditional wall switches, thewall switch 108 has access to plentiful electrical operating power, suchas by connecting to wiring (e.g., to 120V “hot” line voltage wires) thatis behind the walls 154 of the smart-home environment 100 and that ispresent at virtually all standard home wall light switches. Thisessentially unlimited power budget, the near-ubiquity of excellentinstallation locations throughout the home, and the reasonably amplephysical space on its face plate and in its housing to fit the necessarycomponents, combine to enable the wall switch 108 to accommodate a richand wide variety of sensing, interface, and communications capabilitiesfor the smart-home environment 100, as well as a general platform forthe provision of even newer sensing, interface, and communicationshardware as it becomes available over the coming decades.

A rich variety of new interactions are made possible between wall switch108 and other devices of the smart-home environment 100. Occupancysensing, for example, can be significantly enhanced by virtue of thegreat locations (usually right next to room doorways) of most wall lightswitches, allowing for easy tracking of occupants as they transitionbetween rooms, predictive occupancy algorithms, and so forth.

FIGS. 4A-C illustrate exemplary user interfaces and hardware features ofthe wall switch 108. According to embodiments, at the core of the wallswitch 108 is powering circuitry, including a rechargeable battery, forextracting power as needed from the 120V “hot” line voltage wire. Therechargeable battery can either be used as a conventional back-up sourceor as a reservoir to supply excess DC power if needed for short periods.

As illustrated in FIG. 4A, according to some embodiments, the wallswitch 108 is split into two parts: a head unit 404 and a backplate 408.This bifurcation can increase the success and commercial longevity ofthe wall switches 108 by making them a modular platform consisting oftwo basic components. According to some embodiments, the backplate 408is a permanent interface box (sometimes referred to herein as “dockingstation 408”) that serves as a physical connection into the wall and tothe 120V line voltage wires or other wiring of the smart-homeenvironment 100, and that contains a AC-to-DC powering circuitry 410.When installed, the docking station 408 may resemble a conventionalone-gang or two-gang wall box, except no dangerous high-voltage wiresare exposed to the user. According to some embodiments, docking station408 also includes a cellular wireless interface.

According to some embodiments, the head unit 404 (sometimes referred toherein as “replacement module 404”) actually contains all of thesensors, processors, user interfaces, the rechargeable battery, and soforth. Users can plug and unplug the unit 404 in and out of the dockingstation 408. Many different commercial and functional possibilities forprovisioning, maintenance, and upgrade are possible. For example, afteryears of using any particular head unit 404, a user will be able to buya new version of the head unit 404 and simply plug it into the dockingstation 408. There are also many different versions for the head unit404, such as an extremely low-cost version that is nothing but amotion/occupancy detector and light switch, and then a progression ofincreasingly-capable version, up to and including extremely fancy headunit 404 with small OLED televisions and high-fidelity mini-speakers.Thus, it should be appreciated that the various versions of the headunits 404 can all be interchangeable, with any of them working whenplaced into any docking station 408. This can advantageously encouragesharing and re-deployment of old head units 404—for example, when animportant high-capability head unit 404 (for the kitchen or living room,for example) can replaced by a great new version of the head unit 404,then the old head unit 404 can be re-deployed in a bedroom or abasement, etc. When first plugged into a docking station 408, the headunit 404 can ask the user (by 2D LCD display, 2D/3D holographicprojection, voice interaction, etc.) a few simple questions such as,“Where am I” and the user can select “bedroom” or “living room” and soforth.

According to some embodiments, the head unit 404 contains a mainprocessor 412, storage 416, display and user interface 424, audiospeaker 436, microphone 444, power converter 440, GPS receiver 450, RFIDlocater 454, and general physical module receiver 458. The head unit 404further contains wireless and wired networking 462. In view of the amplepower availability, a variety of communications capabilities can beprovided, including Wi-Fi, ZigBee, 3G/4G wireless, CAT6 wired Ethernet,and even optical fiber from the curb. Furthermore, because the wallswitch 108 can be connected to the home 120V system, a HomePlug or otherpowerline-communications capability can be provided.

Also included are sensors 428 such as temperature, humidity, occupancy,ambient light, fire, smoke, carbon monoxide, active proximity, passiveinfrared motion, ultrasound, CCD/video camera, etc. A rechargeablebattery 432 is also included (or equivalently capable onboard powerstorage medium). For example, the battery 432 can be rechargeableLithium-Ion battery. In operation, the wall switch 108 charges thebattery 432 during time intervals in which the hardware power usage isless than what power stealing can safely provide, and that willdischarge to provide the needed extra electrical power during timeintervals in which the hardware power usage is greater than what powerstealing can safely provide.

The user interface 424 can include one or more visual displays (TFT,OLED, etc.), touchscreen and/or button input capabilities, the audiospeaker 436, and so forth. According to the embodiment illustrated inFIG. 4B, the module head unit 404 of smart wall switch 108 has aclick-and-rotate annular ring input 460. According to this embodiment,the click-and-rotate annular ring input 460 of the wall switch 108 canbe used as a dimming light switch. Further, the click-and-rotate annularring input 460 can provide menu-driven interfaces for user governance ofall its various capabilities. As illustrated in FIG. 4C, an optional 2Dimage and/or 3D holographic image projector 470, can also be provided sothat the effective dimension of the display is not just limited to thephysical size of the wall light switch. For example, in smart-homeenvironments 100 that also include smart doorbells 106, the combinationof the smart wall switches 108 and the smart doorbells 106 can bringabout new features. For example, if a visitor approaches the front dooror rings the doorbell 106, the camera in the doorbell 106 can transmittheir image to the smart wall switches 108, which can instantly show theimage on the dial of the light switch, or project the image in 2D or 3D(holographic) form. Further, for example, the image of the arrivingvisitor at the front door can be projected only from the smart wallswitches 108 of those rooms where occupancy is sensed.

According to one embodiment that is particularly appealing for alighting control functionality of the described the wall switches 108,which are temporarily simply termed ‘light switches’ for thislighting-control-related embodiment. The light switches in the home,such as the smart-home environment 100, are configured such that atleast one of them can be used to control (a) all of the light switchesin the home, (b) a single selectable one of the other light switches inthe home, and/or (c) one or more selectable groups or sub-groups of theother light switches in the home. This can be particularly convenient,for example, for a light switch that is near the front door (or otherentryway commonly used for ingress and egress) of the home. For suchexemplary scenario, an occupant who is the last person to leave the homecan simply turn off all lights at once by controlling the single lightswitch nearest the door. Similarly, a returning occupant can turn on aparticular subset of lights (for example, the lights corresponding to apathway from the front door to the kitchen) all at once by controllingthat single light switch. As another advantageous example, a lightswitch that is in an upstairs master bedroom of the home can be used bythe parents to control (a) the lights in all downstairs rooms in thehome, (b) the lights leading between the master bedroom and the kitchen,and/or (c) the lights in each of the children's bedrooms.

The above-described provision for mutually controllable or selectivelymutually controllable light switches can be extended in otherembodiments to similar mutual or selectively mutual control of anypopulation of smart-home controllers that are so amenable. Examples caninclude mutual or selectively mutual control of irrigation controllers,door opening/closing actuating controllers, entertainment devicecontrollers, computing device controllers, portable plug-in heatercontrollers, window opening/closing controllers, and so on.

According to some embodiments, the above-described provisions formutually controllable or selectively mutually controllable smart-homedevices can be further enhanced by artificial intelligence algorithmscapable of making rules-based or learning-based inferences based onsensed patterns of user control of smart-home devices, optionally inconjunction with sensed home conditions or other sensed user behaviors.By way of example, for one embodiment, one or more of the light switchesis configured to process information acquired by home occupancy sensingdevices in conjunction with information from light switch controlbehaviors of the user to produce a result in which the light switchesare automatically self-programmed to help the user turn off all theirhome lights before leaving the home. More particularly, the artificialintelligence algorithms can be configured to sense whether there havebeen a threshold number of weekdays over the past month (or otherevaluation period) for which, at roughly the same time of day (“Xo'clock”), (i) the user has turned off a same or roughly same set ofactive light switches in the home, including the front door lightswitch, over a five-minute period (or other pre-departure period), and(ii) this event was followed by an extended period of non-occupancy,such as at least one hour of non-occupancy. If such pattern has beendetected, the user can be sent a message on their smartphone allowingthem to opt-in to a setting in which, if the light switch near the frontdoor is turned off at about X-o'clock on a weekday, the set of activelight switches will be automatically turned off as well.

By way of example, the opt-in message can say, “If you turn off thelight switch near the front door at about X o'clock on weekdays, I canautomatically turn off the rest of the lights for you that you have beenturning off manually at about that time. It might save you a few steps!Would you like me to do this for you? [YES/NO]′ The user can then opt into this functionality. If the user's schedule changes and they end upnot wanting this functionality later on, the artificial intelligencealgorithms can readily ‘unlearn’ it by virtue of detecting a ‘punishing’or ‘undo’ action by the user in which, the next time the user turns offthe front door light switch, they simply walk to one of theautomatically-turned-on light switches and turn it off within fiveminutes (or other undo interval). Thus, in accordance with the describedembodiment, the collection of smart-home sensors is configured toautomatically learn selected user behaviors, to assist the user withcertain tasks when selected qualifying behaviors are observed, and toautomatically stop providing such assistance when one or morestraightforward undo actions are taken by the user.

Turning now to FIG. 5, an illustration is provided of an exemplaryembodiment of the smart hazard detector 104. According to embodiments,an important underlying functionality of each smart hazard detector 104is for smoke detection, fire detection, and carbon monoxide detection(more generally, “hazard detection”) and associated audible alarming,via a speaker 504 and a buzzer 508. However, it should be appreciatedthat the smart hazard detector 104 is further enhanced withnetwork-connectedness and a variety of multi-sensing capabilities that,while indeed enhancing home safety and security in many ways, canprovide additional functionalities relating to HVAC control, home energyconservation, intra-home communications, and entertainment.

According to embodiments, the smart hazard detector 104 is a retrofitdesigned to replace older hazard detectors. Although not shown in thefigures, the smart hazard detector 104 may comprise two primarycomponents: a head unit 404 and a backplate or docking station 408. Thehead unit comprises the sensors, battery, processor, storage, and othercomponents, while the docking station serves as a physical connectioninto the wall and, if applicable, to the 120V line voltage wires orother wiring of the smart-home environment 100. When installed, thedocking station may resemble a conventional backplate for a traditionhazard detector. Users can plug and unplug the head unit in and out ofthe docking station. Thus, many different commercial and functionalpossibilities for provisioning, maintenance, and upgrade are possible.For example, after years of using any particular head unit, a user willbe able to buy a new version of the head unit and simply plug it intothe docking station. There are also many different versions for the headunit 404, such as an extremely low-cost version that is nothing but asmoke detector, and then a progression of increasingly-capable version,up to and including extremely fancy head units. Thus, it should beappreciated that the various versions of the head units can all beinterchangeable, with any of them working when placed into any dockingstation. This can advantageously encourage sharing and re-deployment ofold head units—for example, when an important high-capability head unit(for detecting hazards in the kitchen, for example) can replaced by anewer version, then the old head unit can be re-deployed in a bedroom ora basement, etc. When first plugged into a docking station, the headunit can ask the user (by 2D LCD display, 2D/3D holographic projection,voice interaction, etc.) a few simple questions such as, “Where am I”and the user can select “bedroom” or “living room” and so forth.

There can be substantial overlap between the smart hazard detector 104and the wall switch 108, as well as the other devices of the smart-homeenvironment 100 with respect to processing, sensing, user interface, andcommunications capabilities. The smart hazard detector 104 may be alow-power consuming device that is powered by battery 512 and thatincludes a low-power communication chip (such as a ZigBee chip) and mayparticipate as a low-power node in the mesh network of the smart-homeenvironment 100 by generating and transmitting messages, relay messagesfrom other devices, as well as by “listen” and sometime making acorresponding response. However, it should be appreciated that insteadof or in addition to being battery powered, the smart hazard detector104 may be powered by AC voltage from the home. In some embodiments, thesmart hazard detector 104 includes a WiFi chipset that enable it tocommunicate its status to other devices in the smart-home environment100, to user mobile devices 166, to the central server orcloud-computing system 164, as well as to external managed securityservices. It should be appreciated that smart hazard detector 104 ismicroprocessor driven and that the WiFi chip may contain extraprocessing capacity for controlling all or some operations of the smarthazard detector 104. In some embodiments, a separate processor isprovided.

According to the illustrated embodiment, the smart hazard detector 104includes a smoke detector 516, which comprising a smoke photodiode, adetector, and a smoke chamber. The smoke photodiode may be, for example,an IR LED. Alternatively, instead of IR, a visible light LED or a lasermay be provided. In some embodiments, the detector may be a photonsilicon photomultiplier chip. The smoke photodiode and the detector maybe programmed to execute a duty cycle every ten seconds, or so. Forexample, every ten seconds the photodiode and the detector will do anX-axes microsecond smoke test. The photodiode and the detector alsoexecute self-calibration tests at predetermined intervals.

Further, the smart hazard detector 104 includes a carbon monoxide sensor520, which can be either an electrochemical sensor or a metal oxidesemiconductor. Additionally, the smart hazard detector 104 may include atemperature sensor, a humidity sensor for reducing false alarmsassociated with showering and cooking, and an ambient light sensor, suchas a single pixel that measures the brightness of the room.

The smart hazard detector 104 may be provided with occupancy detectingcapabilities. In some embodiments, one or more ultrasonic sensors 524are provided for motion detecting. However, it should be appreciatedthat in addition to or instead of the ultrasonic sensors 524, one ormore passive IR sensors 528 are provided for occupancy sensing. Havingmultiple ultrasonic sensor 524 and/or passive IR sensors 528 enhance theoccupancy sensing capabilities of the detector. Because they are typicalmounted in unobstructed locations, high on walls of often-occupiedrooms, smart hazard detectors 104 are particularly well suited foroccupancy detection, such as by the use of RFID, ultrasonic sensors,etc. The smart hazard detector 104 may also include a thermopile 534 forflame-heat detection. As it is well known in the art, the thermopile orthermo-camera 534 is a group of thermo couples that take infrared lightand correlate that to flame heat. In some instances, this isadvantageous because the thermo-camera looks into the room and givesadvanced warning of heat. Thus, the thermo-camera 534 is able to “see”heat before the heat actually makes its way to the smart hazard detector104.

According to embodiments, technologies including the sensors of thesmart hazard detector 104 in combination with rules-based inferenceengines or artificial intelligence provided at a central server such as164 are used to reduce the number of false alarms. For example,inferences about the occupants' activities can be learned based on datareceived over time. For example, if the smart hazard detector 104located in the kitchen observes increased temperature, humidity, andmotion in the kitchen, then an inference can be made that one or more ofthe occupants are cooking (“cooking inference”). These data inputs canbe considered on a sliding scale based on time of day and day of week.For example, only slight increases temperature and humidity may invokethe cooking inference at 5 pm on weekdays, when history shows that whenthe occupants typically cook. However, much larger increases arerequired to invoke the cooking inference at 10 am on a weekday, when theoccupants are typically away at that time. In some embodiments, when thecooking inference is made, the smart hazard detector 104 in the kitchenbecomes less sensitive, whereas the hazard detectors 104 in other roomremain in normal operation.

In some embodiments, the smart hazard detector 104 is equipped with oneor more air quality sensors 538. For example, the air quality sensors538 may “sniff” for volatile organic compounds (VOCs) that may presentin the house. Thus, the smart hazard detector 104 can warn users whenthere are toxins in the user's home, such as when the user is burning aparaffin wax candle. Paraffin is a by-product of petroleum has beenshown to release an alarming range of (VOCs), such as such as tolueneand benzene. Also, for example, the air quality sensors 538 can “sniff”for gas leaks in the house, such as by detecting methane, which iscommonly added to natural gas so as to make natural gas detectable. This“sniffing” capability is not only good for houses, but also for schoolsand hospitals. Individual will find peace of mind knowing such detectionis in widespread use. Further, as houses get more and more sealed, airquality detection will grow in importance. Further, the air qualitysensors 538 can measure levels of particulate, dust, pollen, mold, etc.detection.

In addition to measuring air quality and detecting harmful VOCs, thesmart hazard detector 104 is capable of transmitting this information tothe other devices in the home as well as the central server orcloud-computing system 164. For example, in some embodiments, when thereare elevate levels of patent harmful gases, particulate, dust, pollen,mold, etc. in the home, the smart hazard detector 104 can transmit thisinformation to the central server or cloud-computing system 164, whichcommunicates with other nodes in the home and just outside of the home,such as the smart doorbell 106, and assess whether the inside air or theoutside air is purer. If the outside air is purer, then the centralserver or cloud-computing system 164 instructs the smart thermostat 102to open a vent to permit fresh air into the home, otherwise it instructsthe thermostat 102 to recirculate air in the home and to not draw inoutside air. Furthermore, the central server or cloud-computing system164, upon receiving the air quality information from the hazard device104, can send detailed information about the air quality to the mobiledevice 166 of user. For example, the air quality information mayidentify the specific types of toxins, particulate, dust, pollen, mold,etc. in the air. This can help the user identify which pollen, etc. theuser is allergic to. Further, the central server or cloud-computingsystem 164 can use the air quality information to provide the user withmedication recommendations (e.g., be sure to take you allergy pillstoday). Furthermore, the central server or cloud-computing system 164can aggregate data received from multiple home in various geographiclocations and provide, for example, smog alerts, pollen warnings, etc.

According to some embodiments, the smart hazard detector 104 may includea carbon dioxide (CO₂) sensor for fire detection, where detectorincludes a passive IR detector from lead selenide. Fire produces CO,CO₂, and infrared light. The intensity of the infrared light emitted bythe flame is pretty constant, until it passes through the layer of CO₂produced by the fire. However, the frequency of light having awavelength of 4.26μ excites CO₂ molecules when the light contacts theCO₂. In response, the CO₂ absorbs the infrared light having a wavelengthof 4.26μ, thereby decreasing the intensity of the light at thiswavelength. Thus, to detect fire, a narrow optical bandpass filter isplaced in front of the lead selenide IR detector to permit only lighthaving a wavelength 4.26μ to pass through to the lead selenide IRdetector. In the event of fire, the lead selenide detects the decreasein intensity of light at a wavelength of 4.26μ and triggers the firealarm.

According to embodiments, technologies including the sensors of thesmart hazard detector 104 in combination with rules-based inferenceengines or artificial intelligence provided at a central server such as164 are used to warn occupants of dangerous conditions in the home. Forexample, an inference can be made that an occupant is burning a candlebased of the qualities (e.g., intensity, wavelength, frequency, etc.) ofthe light and/or the toxins begin released into the air. Further, aninference can be made that the occupant in the room has fallen asleepwhen the occupant is in the room and has not moved for a predeterminedperiod of time. In situations where a candle is burning and the occupanthas fallen asleep, the smart hazard detector 104 in that room will soundan alarm to wake up the occupant. In other example, the smart hazarddetector 104 can sound an alarm or send an alert text or email messagewhen it detects dangerous conditions in unoccupied rooms, such as whenthe stove is left on in the kitchen.

In the event a fire does occur in the home, technologies including thesensors of the hazard detectors 104 in combination with rules-basedinference engines or artificial intelligence provided at a centralserver such as 164 are used to determine the cause. For example, thesmart hazard detector 104 can detect that the fire started in thekitchen, and the smart hazard detector 104 can detect an active stove,candle, etc. was left unattended in the kitchen just before the firestarted. Additionally, a central server such as 164 can aggregate datafrom multiple homes that experienced a fire, analyze the data to findpatterns and reconstruct what caused the fires, and share thisinformation with fire departments.

In some embodiments, the smart hazard detector 104 is mounted in alocation that is out-of-reach of the occupants, such as high on a wallor on a ceiling. Thus, in the event of a false alarm, the occupantscannot reach the smart hazard detector 104 to press a button thatdeactivates the alarm. Accordingly, embodiments of the smart hazarddetector 104 allows for occupants to deactivate the alarm using “silencegestures”. For example, the smart hazard detector 104 detects aparticular “silence gesture” from one of the occupants and, in response,deactivates the alarm.

According to embodiments, the one or more ultrasonic sensors 524 of thesmart hazard detector 104 are used to detect the “silence gesture” ofthe occupant. In some embodiments, each of the ultrasonic sensors 524may contain multiple piezos so as to make the sensor “omni-directional”and to make it easier to detect when an occupant is making a silencegesture. Using ultrasonic sensors 524 in battery-powered versions of thesmart hazard detector 104 is particularly advantageous becauseultrasonic sensors 524 consumer very little power. The ultrasonicsensors 524 work by sending out pulses and then calculating the timeinterval between sending the pulse and receiving a corresponding echo todetermine the distance to an object. In some embodiments, a single piezoof the ultrasonic sensor sends the pulse and receives the echo. In otherembodiments, separate piezos send pulses and receive echoes.

During normal operation, the ultrasonic sensors 524 of the smart hazarddetector 104 are merely performing occupancy sensing functions. Forexample, they are determining whether the room is occupied. Whenperforming this function, the ultrasonic sensors 524 have a ping rate ofabout one hertz. This low ping rate is designed to preserve batterypower, while still effectively performing occupancy detection. However,when one of the other sensors, such as the smoke, fire, or carbonmonoxide sensors, activates the alarm, the smart hazard detector 104increases the ping rate, such as up to twenty hertz. The increased pingrate better enables detection of “silence gestures”, which are only madewhen the alarm is active. When detecting a “silence gesture”, theultrasonic sensors 524 are “looking for” an object, such as a humanhand, to remain within in a predefined distance from the smart hazarddetector 104 for a specified period.

An example “silence gesture” will be described with references to FIGS.6 and 7. As shown in FIG. 6 at block 604, an occupant is standing inroom 612 while an alarm in hazard detector 104 is active and making a“BEEP” sound. A light 610, such as an LED, is provided on an outerportion of the smart hazard detector 104, such that the occupant 608 cansee the light 610 when it is turned on. The operation of the light 610will be described with reference to FIG. 7. Suffice to say for FIG. 6,the light is turned off in blocks 604 through 624. As shown at block616, the occupant 608 has walked to a position closer to the smarthazard detector 104, which is mounted out of reach on the ceiling of theroom. As shown at block 620, the occupant 608 walked to a position evencloser to the smart hazard detector 104, such that the occupant 608 isalmost directly under the smart hazard detector 104. As shown at arrow628 of block 624, the occupant 608, while standing almost directly underthe smart hazard detector 104, is beginning to extend an arm upward,toward the smart hazard detector 104.

Referring now to block 630 of FIG. 7, the arm of the occupant 608 isextended upward, toward the smart hazard detector 104, while theoccupant is standing almost directly under the smart hazard detector104. After an alarm sounds and the pulse rate increases, the ultrasonicsensor the smart hazard detector 104 “looks” for a trigger to the“silence gesture” period, which is the amount of time the “silencegesture” must be maintained to deactivate the alarm. According to someembodiments, the trigger is a distance change from a baseline, and todeactivate the alarm the distance change must be maintained for theentire “silence gesture” period (e.g., three seconds). For example, ifthe baseline is a distance between the sensor and the floor of the room,then the sensor is looking for an object to come in between it and thefloor, thereby changing the distance measured by the sensor. In someembodiments, the distance change must be significant enough to ensurethat someone is close and likely intends to silence the alarm. Forexample, if the distance to the floor is ten feet, then the requisitedistance change could be eight feet or eighty percent of the originaldistance. As such, the object would be required to be within two feet ofthe sensor to trigger the “silence gesture” period, and to deactivatethe alarm, the object must remain there for the duration of the period.The requisite distance change can be configured based on the height ofthe ceiling and based on the height of the occupants, among otherthings.

Referring still to block 630, the light 610 is turned on when theoccupant 608 successfully triggers the “silence gesture” period, therebysignaling to the occupant 608 to remain in the position for therequisite period, such as three seconds. Here, the hand of the occupant608 triggered the “silence gesture” period. A tolerance is built in suchthat if the occupant 608 slightly moves and loses but quickly regainsthe signal, the “silence gesture” period will continue without having tostart over. As shown in block 634, the occupant kept the hand in withinthe requisite distance of the sensor for the duration of the “silencegesture” period and, thus the alarm has been deactivated, the “BEEP” hasstopped, and the light 610 has turned off. As shown at blocks 638 and642, the occupant 608 can walk away from the smart hazard detector 104and resume normal activity.

It should be appreciated that, in the event the smart hazard detector104 is of a design that receives reliable power from the wiring of thehome (rather than being batter powered), a CCD chip could be used todetect the “silence gesture”. However, such an arrangement is notsuitable for battery-powered hazard detectors 104 because the CCD chipsand associated processing consume a large amount of power and wouldquickly drain the battery. Other possible alternatives to ultrasonicsensors 524 include passive IR sensors, thermopile (e.g.,thermo-cameras), laser-distance measuring, laser and a cameracombination because camera looks for dot instead of time of arrival(Doppler shift), and a full on camera and image processing system.

According to some embodiments, to enhance the reliability andeffectiveness of the silence gesture, the ultrasonic sensor 524 couldwork in concert with the passive IR sensor to make the sensing evenbetter. For example, when an occupant attempts to silence by placing ahand in field, the passive IR will sense this, and thereby trigger the“silence gesture” period. The ultrasonic sensor 524 could also work inconcert with the thermopile (e.g., thermo-camera), where both distancechange and heat are used to detect the silence gesture. For example, thethermo-camera detects when human hand is nearby and triggers the“silence gesture” period. Further, the ultrasonic sensor 524 could workin concert with the ambient light sensor. For example, when the places ahand in the field and blocks light, then the ambient light sensor knowthe occupant is nearby and thus triggers the “silence gesture” period.

It should be appreciated that, according to embodiments, similar“gesture” controls can be applied to other smart devices in the home,such as to the smart thermostat, the smart wall switches, etc. Forexample, there can be gestures for increasing or decreasing temperaturecontrols, for turning on and off lights, HVAC, etc.

Turning now to FIG. 8, an illustration is provided of an exemplaryembodiment of a smart doorbell 106. According to embodiments, animportant underlying functionality of the smart doorbell 106 is to serveas a home entryway interface unit, providing a doorbell functionality(or other visitor arrival functionality), audio/visual visitorannouncement functionality, and like functionalities. Like the smarthazard detector 104 described above with reference to FIGS. 5-7, thesmart doorbell 106 is further enhanced with network-connectedness and avariety of multi-sensing capabilities to accommodate additionalfunctionalities, and there can be substantial overlap/integration amongthe smart doorbell 106, the smart hazard detector 104, the smart wallswitch 108, and the smart wall plug 110 that leverages their combinedprocessing, sensing, and communications capabilities, as well as theiraccess to cloud-based control and intelligence.

In some embodiments, the smart doorbell 106 is connected to the wiringof the smart-home environment 100. For example, as is common in manyhomes, a 24V low voltage wire is provided at the outer entry points ofthe home, such as at the front, back, and side doors. The smart doorbell106 can be connected to this 24V low voltage wire to obtain steady andreliable power. However, it should be appreciated that the smartdoorbell could include a battery for the purpose of replacing orsupplementing power obtained from the home wiring. In some embodiments,the battery could be a rechargeable battery, such as a rechargeableLithium-Ion battery, for extracting power as needed from the housewiring (e.g., 24V low voltage wire). For example, the smart doorbell 106could charge the battery during time intervals in which the doorbell'spower usage is less than what the 24V low voltage wire can safelyprovide, and that will discharge to provide the needed extra electricalpower during time intervals in which the hardware power usage is greaterthan what the 24V low voltage wire can safely provide. Thus, therechargeable battery could be used as a conventional back-up source oras a reservoir to supply excess DC power if needed for short periods.

As illustrated in FIG. 8B, according to some embodiments, the smartdoorbell 106 includes two parts: a head unit 804 and a backplate 808,which is also referred to as a docking station 808. This bifurcation canincrease the success and commercial longevity of the wall switches 108by making them a modular platform consisting of two basic components. Aswith the wall switches 108 and the hazard detectors 104, thisbifurcation can increase the success and commercial longevity of thesmart doorbells 106 by making them a modular platform. The dockingstation 808 is a permanent interface box that serves as a physicalconnection into area near the entry way, such as a doorframe or outerwall of the home, and to the voltage wires of the home. According tosome embodiments, the head unit 804 (sometimes referred to herein as“replacement module 804”) actually contains all of the sensors,processors, user interfaces, the rechargeable battery, and so forth.Users can plug and unplug the unit 804 in and out of the docking station808. Many different commercial and functional possibilities forprovisioning, maintenance, and upgrade are possible. For example, afteryears of using any particular head unit 804, a user will be able to buya new version of the head unit 804 and simply plug it into the dockingstation 808. There are also many different versions for the head unit804, such as an low-cost versions with few features, and then aprogression of increasingly-capable versions, up to and includingextremely fancy head units 804 with a large number of features. Thus, itshould be appreciated that the various versions of the head units 804can all be interchangeable, with any of them working when placed intoany docking station 808. This can advantageously encourage sharing andre-deployment of old head units 804—for example, when an importanthigh-capability head unit 804 (for the front door, for example) canreplaced by a great new version of the head unit 804, then the old headunit 804 can be re-deployed to a back or basement door, etc. When firstplugged into a docking station 808, the head unit 804 can ask the user(by 2D LCD display, 2D/3D holographic projection, voice interaction,etc.) a few simple questions such as, “Where am I” and the user canselect “front door” or “back door” and so forth.

Sensors devices such as temperature, humidity, occupancy, ambient light,fire, smoke, carbon monoxide, active proximity, passive infrared motion,ultrasound, CCD/video camera, bar code scanner, etc., as well as I/Odevices such as speakers, user interfaces, 2D/3D projectors, etc. areprovided in the head unit 804. The sensors and I/O devices are generallyrepresented at 828.

According to embodiments, technologies including the sensors 828 incombination with rules-based inference engines or artificialintelligence provided at a central server such as 164 are used to detectwhen packages are delivered to the door of the smart-home environment100, and to take a variety of automated actions in response. Accordingto some embodiments, the sensors 828 can “see” the uniform of thedelivery person approaching the door or the truck of the deliveryperson, or the sensors can “hear” the truck in combination with a personapproaching the door within a period after hearing the truck. Once theperson is with a predetermined distance from the door, the smartdoorbell 106, using its speaker, asks the person is he or she is adelivery person, to which that person can reply with an audible responseor by indicating as such on the user interface of the doorbell. If theperson is making a delivery, the doorbell can instruct the person toplace the package in a location proximate the doorbell 106 such that itsscanner can scan the bar code or other type of identifying tag affixedto or associated with the package being delivered.

According to embodiments, in the A central server, such as server 164,can use the bar code to obtain information about the package, such asthe sender, recipient, a description of the item therein, whethersignature is required, etc. Based on this information the rules-basedinferences engines or artificial intelligence can make inferenceregarding which action to take next. For example, if occupants or moreparticularly if the specified recipient or recipients are in the homeand the package is from a sender or contains an item that is ofparticular interest to the one or more the occupants, an announcementcan be made in the home, indicating the package is currently beingdropped off and providing details about the package. On the other hand,the sender or item is not of particular interest to one of the occupantsbased on the occupants past responses to receiving such deliveries, thenno announcement will be made and the occupants can discover the packagein due course. Similarly, if no occupants are home but an inference ismade that the package is of particular interest to one of the occupants,a text message, email, phone call, etc. can be made the one or moreidentified occupants, indicating that the package in being delivered.

According to embodiments, in the event signature is required for thepackage but none of the occupants are home or an inference is made tonot disturb the occupants, the smart doorbell can provide authorizationto leave the package. For example, the doorbell 106 present anauthorization code (either in plain text, barcode, or encrypted forms)on its user interface and prompt the delivery person to use his or herhandheld device to record, scan, photograph, or other acquire theauthorization code. Once the authorization code has been acquired by thedelivery person, the doorbell 106 can then instruct the delivery personregarding where to leave the package, such as at the front door, aroundthe back of the house in a discrete location, etc.

According to embodiments, technologies including the sensors 828, suchas noise, motion, and/or facial recognition detecting, in combinationwith rules-based inference engines or artificial intelligence providedat a central server, such as server 164, are used to detect when one ormore unknown individuals are approaching the home. A number of learnedinferences can be made in this situation. For example, during lateevening hours, an alarm will be sounded in the house, giving notice ofthe approaching person and/or the doorbell 106 may audible announce tothe individual that he or she is being monitored by a home securitysystem. Further, in the event the person attempt to enter a door,window, or other access point to the home, an message will be sent tolocal law enforcement. However, during day time hours when one or moreoccupants are at home, a learned inference can be made to take noaction, such as when a meter reader, mail carrier, garbage collector,etc. is approaching the house as per a reasonably predictable andlearned schedule.

According to embodiments, when the central server such as server 164,based on information received from the doorbell 106, such as noise andmotion data, determines that the street adjacent to the home has athreshold level of traffic, the inferences are made regarding the safeof the children residing at the home. For example, an alarm can betriggered in the home in the event one or more of the children aredetected by the occupancy sensing to be outside of the home. This alertenables the parents or other caretakers to quickly take actions toprotect the child from the traffic. Further, for example, automaticadjustments are made to audio equipment in the home of account for theincreased traffic noise, such as by increasing the volume aproportionate amount.

As discussed, the sensors 828 may include temperature and humiditysensors, the data from which may be used for a number of usefulservices. For example, the outside humidity and temperature data isconsider by the thermostat 102 when controlling the HVAC to bestaccomplish the occupants' desired comfort preferences. Further, forexample, this information may be presented to the occupants through anumber of user interfaces, such as a user interface associated withanother one of the devices located inside of the home, the television,mobile and other computing devices, or audibly. In some instances, acentral server such as server 164 collects this information from aplurality of smart home across a plurality of geographic locations. Thisaggregated data may be sold to weather services or may be used toprovide weather data to smart home occupants.

According to embodiments, the smart doorbell 106 includes a button 812that, upon being touched, depressed, or otherwise activated, causes asaudible notification to be broadcasted within the home or a message tobe sent to user interfaces of devices within the home or to a mobiledevice associated with occupants of the home. Learned inferences can bemade regarding the appropriate response to activation of the button 812.For example, the audible notification is only broadcast in occupiedrooms, or rooms occupied by one or more occupants who have arelationship with the person at the door, or no alarm is sound in roomswhere occupants, such as small children, are determined to be sleeping.Also, for example, the occupant-selected songs, such as uploaded MP3's,may be broadcasts in the home, where different songs may be broadcastfor different occupants at home at the time or based on the identity ofthe person at the door. Further, for example, technologies and sensorsat the smart doorbell 106 may identify the person based on facialrecognition or based on other characteristics such as the manner inwhich the person approached the door. For example over time, based oninput received from the smart doorbell 106 a central server can build upan address book of profile data about people who approach the door, suchas some identifying biometric data. For example, the address book can bebuilt over time using low-resolution data such as ultrasonic, passiveIR, etc. to create a unique signature for individuals. This combineddata from different domains and becomes almost like a fingerprintregarding how that person approaches the house. In some instances, whena “familiar” person approaches the door, the smart doorbell 116 “asks”the person if he is “John Doe”, to which the person can verbally orphysically respond. Upon obtaining this information John Doe's name orimage can be announced or projected on device in the home and/or JohnDoe will be given certain access rights to the home, such as for examplethe door will automatically unlock as he approaches. Further, inaddition to or instead of identification based on these unique“signatures”, individuals may enable their mobile devices to communicatewith the smart doorbell 116, such as via Bluetooth, NFC, or otherwireless protocols. Also, for example, individual may “swipe” theirsmart phones in front of the smart doorbell's RFID scanner. Uponidentifying the individual, the smart doorbell may give the individualcertain access right to the home, such as by automatically unlocking thedoor.

According to embodiments, technologies including the sensors 828 incombination with rules-based inference engines or artificialintelligence provided at a central server, such as server 164, also makelearned

Turning now to FIG. 9, an illustration is provided of an exemplaryembodiment of a smart wall plug 110. According to embodiments, the smartwall plugs 110 enhance the smart-home environment 100 by providing aretrofit wall plug that, in addition to maintaining the basic characterand purpose of a wall plug, incorporates a host of host of sensing andcommunications capabilities for enhancing occupant comfort, convenience,and safety. Much like the smart wall switches 108, by virtue of beingmounting in place of traditional wall plugs, the smart wall plugs 110have access to plentiful electrical operating power, such as byconnecting to wiring (e.g., to 120V “hot” line voltage wires) that isbehind the walls 154 of the smart-home environment 100 and that ispresent at virtually all standard home wall plugs. This unlimited powerbudget, the numerous installation locations throughout the home, and thereasonably ample physical space in its housing to fit the necessarycomponents, combine to enable the smart wall plugs 110 to accommodate arich and wide variety of sensing and communications capabilities for thesmart-home environment 100, as well as a general platform for theprovision of even newer sensing and communications hardware as itbecomes available.

A rich variety of new interactions are made possible between the smartwall plugs 110 and other devices of the smart-home environment 100.Occupancy sensing, for example, can be enhanced by virtue of thenumerous installation locations of wall plugs through the home, albeitsome of these locations are hidden behind furniture. According toembodiments, the smart wall plugs 110 can include all or some of thecomponents that are included in the wall switch 108, as illustrated inFIGS. 4A-C. For example, the smart wall plugs 110 can be comprised oftwo primary parts: a head unit 908 and a docking station 904, which is apermanent interface box that serves as a physical connection into thewall and to the 120V line voltage wires or other wiring of thesmart-home environment 100. This bifurcation can increase the successand commercial longevity of the smart wall plugs 110 by making them amodular platform consisting of two basic components. According toembodiments, the head unit 908 contains the sensors, processors, the I/Odevices and receptacles, and so forth. Users can upgrade and/orredistribute wall plug functionality throughout the home by plugging andunplugging head units of varying capabilities in and out of the dockingstations in various locations throughout the home. For example, headunits 908 having advanced occupancy sensing can be plugged into dockingstations located in area having clear lines of sight in commonlyoccupied rooms, rather than behind furniture in or unused rooms. Also,for example, head units having advance communication capabilities areplugged into docking stations associated with high-energy consumingappliances, such as a clothes dryer. As discussed, these head units maybe capable of wirelessly communicating with smart meters andcorresponding providing power to the clothes dryer at off-peak hours.Because this head unit may be located behind a clothes dryer in ararely-occupied laundry room, it is unnecessary for this head unit toinclude advanced occupancy sensors, for example.

According to some embodiments, the head unit 908 includes a standardduplex receptacle 960 having upper and lower outlets, each having threeslots. The head unit 908 further contains some or all of a mainprocessor, storage, audio speaker, microphone, power converter, GPSreceiver, RFID locater. Additionally, The head unit 908 may includewireless and wired networking. In view of the ample power availability,a variety of communications capabilities can be provided, includingWi-Fi, ZigBee, 3G/4G wireless, CAT6 wired Ethernet, and even opticalfiber from the curb. Furthermore, because the smart wall plug 110 can beconnected to the home 120V system, a HomePlug or otherpowerline-communications capability can be provided (See FIGS. 4A-C forexample illustrations of such components being provided on a wall switch108). It should also be appreciated that the smart wall plugs 110 mayinclude sensors 928 such as temperature, humidity, occupancy, ambientlight, fire, smoke, carbon monoxide, active proximity, passive infraredmotion, ultrasound, CCD/video camera, etc.

According to embodiments, the smart wall plugs 110 wirelesslycommunicate with smart meters, which are electrical meters that recordconsumption of electric energy in time intervals and then transmits thatinformation a local radio receiver, which forward the information toutility companies. Utility companies use this data to charge varyingrates. For example, power used midday during peak hours is moreexpensive than energy used at night during off-peak hours. The smartwall plugs 110 communicate with the smart meter of the home to obtainpower-usage data. This data can be forwarded to a central server such asserver 164. Rules-based inference engines or artificial intelligenceprovided at a central server such as 164 to make decisions regarding howto efficiently operate certain appliances in the home, so as to reduceenergy costs, yet also satisfy the occupants' preferences. For example,decisions are made regarding use appliances for non-essential tasks atnight (e.g., wash dishes at night), and to permit essential tasks whennecessary (e.g., allow refrigerator compressor to turn on mid-day).Furthermore, a central server can be programmed with safe operatingparameters for each application in the smart home, and can shutoff powerto the appropriate wall plug 110 when the corresponding applicationdeviates from its safe operating parameters. Furthermore, theintelligence of the central process can be used for safety. For example,in the event data from an occupancy sensor on a particular wall plug 110indicates that a small child is really close the wall plug, the centralserver may shut off power to that plug. Further, as a general rule, thecentral server may shut off power to all wall plugs that do have anappliance other type of power consuming device plugged into it. Further,user can remotely control, such as by use of their mobile device, theoutlets in the home. For example, a parent may want to shutoff power toall wall plugs 110 in their child's room, or that are outside of thehome and there is some risk that a child may be attracted to the plug.

Turning now to FIGS. 10A-B, illustrations of a smart thermostat 102 areprovided, according to some embodiments. Unlike many prior artthermostats, smart thermostat 102 preferably has a sleek, simple,uncluttered and elegant design that does not detract from homedecoration, and indeed can serve as a visually pleasing centerpiece forthe immediate location in which it is installed. Moreover, userinteraction with smart thermostat 102 is facilitated and greatlyenhanced over known conventional thermostats by the design of smartthermostat 102. The smart thermostat 102 includes control circuitry andis electrically connected to an HVAC system, such as is shown with unit100 in FIGS. 1 and 2. Smart thermostat 102 is wall mounted, is circularin shape, and has an outer rotatable ring 1012 for receiving user input.Smart thermostat 102 is circular in shape in that it appears as agenerally disk-like circular object when mounted on the wall. Smartthermostat 102 has a large front face lying inside the outer ring 1012.According to some embodiments, smart thermostat 102 is approximately 100mm in diameter.

The outer rotatable ring 1012 allows the user to make adjustments, suchas selecting a new target temperature. For example, by rotating theouter ring 1012 clockwise, the target temperature can be increased, andby rotating the outer ring 1012 counterclockwise, the target temperaturecan be decreased. The smart thermostat 102 may be configured to receivea plurality of types of inputs by virtue of the rotatable ring 1012,such as a scrolling input and a selection input. For example, a rotationof the ring may allow a user to scroll through an array of selectionoptions, and inwards pressure exerted on the ring (inward click) mayallow a user to select one of the options (e.g., corresponding to aparticular scroll position).

The outer rotatable ring 1012 may include a component that may bephysically rotated, or, in other embodiments, a static component thatmay sense a user's virtual rotation of the ring. For some embodiments,the outer rotatable ring 1012 may include a touch pad configured totrack arcuate motion of a user's finger on the touch pad. The touch padmay comprise, e.g., a ring-shaped or circular area. In some instances,the touch pad includes multiple portions (e.g., to detect arcuate motionin a first ring-shaped area and to detect tapping in a second innercircular area). Boundaries of a touch pad area may be identified to auser using, e.g., visual or tactile cues. For example, a ring-shapedtouchpad area may be indented compared to neighboring areas on the smartthermostat 102, or the area may be a different color than neighboringareas.

For preferred embodiments such as those of FIG. 10A in which the outerring 1012 is a continuous loop without fiducial markers, one or moreadvantages are brought about. Thus, a user may physically rotate thering (in embodiments in which the ring is configured to be physicallyrotatable) regardless of a starting position of the ring. Further, auser may select, e.g., a value of a variable (e.g., select a particularmenu, a particular setpoint temperature value, etc.) by rotating thering multiple times. This feature may be particularly advantageous asthe user need not worry about precise rotations in order to select adesired option.

The front face of the smart thermostat 102 comprises a clear cover 1014that according to some embodiments is polycarbonate, and a metallicportion 1024 preferably having a number of slots formed therein asshown. According to some embodiments, the surface of cover 1014 andmetallic portion 1024 form a common outward arc or spherical shapegently arcing outward, and this gentle arcing shape is continued by theouter ring 1012.

Although being formed from a single lens-like piece of material such aspolycarbonate, the cover 1014 has two different regions or portionsincluding an outer portion 1014 o and a central portion 1014 i.According to some embodiments, the cover 1014 is painted or smokedaround the outer portion 1014 o, but leaves the central portion 1014 ivisibly clear so as to facilitate viewing of an electronic display 1016disposed thereunderneath. According to some embodiments, the curvedcover 1014 acts as a lens that tends to magnify the information beingdisplayed in electronic display 1016 to users. According to someembodiments the central electronic display 1016 is a dot-matrix layout(individually addressable) such that arbitrary shapes can be generated,rather than being a segmented layout. According to some embodiments, acombination of dot-matrix layout and segmented layout is employed.According to some embodiments, central display 1016 is a backlit colorliquid crystal display (LCD). An example of information displayed on theelectronic display 1016 is illustrated in FIG. 10A, and includes centralnumerals 1020 that are representative of a current setpoint temperature.

Particular presentations displayed on the electronic display 1016 maydepend on detected user input. For example, one of a plurality ofvariables (e.g., current setpoint temperature versus learning status) orvariable values (e.g., 65 degrees versus 75 degrees) may be displayed.The one being displayed may depend on a user's rotation of the outerrotatable ring 1012. Thus, for example, when the device is configured todisplay a current setpoint temperature, the value being displayed maygradually increase as the user rotates the ring in a clockwisedirection. The sign of the change in the displayed temperature maydepend on whether the user is rotating the ring in a clockwise orcounterclockwise direction. The speed at which the displayed temperatureis changing may depend (e.g., in a linear manner) on the speed at whichthe user is rotating the ring.

As described above, a displayed characteristic may vary depending onreceived user input. For example, a displayed temperature may increaseas a user rotates the outer rotatable ring 1012 clockwise, or ahighlighted indicator may progress across a list of displayed options asthe user rotates the ring 1012. Further, or additionally, user inputsmay cause the appearance of new types of information. For example, if auser is viewing setpoint-temperature options, a dramatic clockwiserotation may cause a flashing red symbol (to convey ananti-environmental message). Thus, a relationship may exist between asingle type of user input (e.g., ring rotation) and a change in anactive variable (e.g., setpoint temperature changes), and relationshipsmay further exist between the single type of user input and an inactivevariable (e.g., an environmental warning flag). The latter relationshipmay be indirect and depend on a value or change in values of the activevariable.

The presentations on the electronic display 1016 may depend on one ormore types of user input. For example, the display may change in a firstmanner (e.g., to show a varying selection option) as a user rotates theouter rotatable ring 1012 and may change in a second manner (e.g., toconfirm a selection or default to a menu screen) as the user exertsinwards pressure on the outer rotatable ring 1012.

According to some embodiments, metallic portion 1024 has number ofslot-like openings so as to facilitate the use of a passive infraredmotion sensor 1030 mounted therebeneath. The metallic portion 1024 canalternatively be termed a metallic front grille portion. Furtherdescription of the metallic portion/front grille portion is provided inthe commonly assigned U.S. Ser. No. 13/199,108. The design of themetallic portion 1024 compliments the sleek, simple, uncluttered andelegant design of smart thermostat 102 while facilitating theintegration and operation of sensors located within a housing of thethermostat. In the implementation as illustrated, smart thermostat 102is enclosed by housing with a forward-facing surface including the cover1014 and the metallic portion 1024. Some implementations of the housinginclude a back plate and a head unit. The housing provides an attractiveand durable configuration for one or more integrated sensors used bysmart thermostat 102 and contained therein. In some implementations, themetallic portion 1024 may be flush-mounted with the cover 1014 on theforward-facing surface of housing. Together the metallic portion 1024 asincorporated in housing does not detract from home or commercial decor,and indeed can serve as a visually pleasing centerpiece for theimmediate location in which it is located.

The metallic portion 1024 is designed to conceal sensors from viewpromoting a visually pleasing quality of the thermostat yet permittingthem to receive their respective signals. Openings in the metallicportion 1024 along the forward-facing surface of the housing allowsignals to pass through that would otherwise not pass through the cover1014. For example, glass, polycarbonate or other similar materials usedfor cover 1014 are capable of transmitting visible light but are highlyattenuating to infrared energy having longer wavelengths in the range of10 microns, which is the radiation band of operation for many passiveinfrared (PIR) occupancy sensors. Notably, included in the smartthermostat 102, according to some preferred implementations, is anambient light sensor (not shown) and an active proximity sensor (notshown) positioned near the top of the thermostat just behind the cover1014. Unlike PIR sensors, the ambient light sensor and active proximitysensor are configured to detect electromagnetic energy in the visibleand shorter-infrared spectrum bands having wavelengths less than 1micron, for which the glass or polycarbonate materials of the cover 1014are not highly attenuating. In some implementations, the metallicportion 1024 includes openings in accordance with one or moreimplementations that allow the longer-wavelength infrared radiation topass through the openings towards a passive infrared (PIR) motion sensor1030 as illustrated. Because the metallic portion 1024 is mounted overthe radiation receiving surface of PIR motion sensor 1030, PIR motionsensor 1030 continues to receive the longer wavelength infraredradiation through the openings and detect occupancy in an enclosure.

Additional implementations of the metallic portion 1024 also facilitateadditional sensors to detect other environmental conditions. Themetallic portion may at least partly conceal and/or protect one or moresuch sensors. In some implementations, the metallic portion 1024 helps atemperature sensor situated inside of the thermostat's housing measurethe ambient temperature of air. Openings in the metallic portion 1024promote air flow towards a temperature sensor located below the metallicportion 1024 thus conveying outside temperatures to the interior of thehousing. In further implementations, the metallic portion 1024 may bethermally coupled to a temperature sensor promoting a transfer of heatfrom outside the housing.

The smart thermostat 102 is preferably constructed such that theelectronic display 1016 is at a fixed orientation and does not rotatewith the outer ring 1012, so that the electronic display 1016 remainseasily read by the user. For some embodiments, the cover 1014 andmetallic portion 1024 also remain at a fixed orientation and do notrotate with the outer ring 1012. According to one embodiment in whichthe diameter of the smart thermostat 102 is about 80 mm, the diameter ofthe electronic display 1016 is about 45 mm. According to someembodiments an LED indicator 1080 is positioned beneath portion 1024 toact as a low-power-consuming indicator of certain status conditions.For, example the LED indicator 1080 can be used to display blinking redwhen a rechargeable battery of the thermostat is very low and is beingrecharged. More generally, the LED indicator 1080 can be used forcommunicating one or more status codes or error codes by virtue of redcolor, green color, various combinations of red and green, variousdifferent blinking rates, and so forth, which can be useful fortroubleshooting purposes.

Motion sensing as well as other techniques can be use used in thedetection and/or predict of occupancy, as is described further in thecommonly assigned U.S. Ser. No. 12/881,430, supra. According to someembodiments, occupancy information is used in generating an effectiveand efficient scheduled program. Preferably, an active proximity sensor1070A is provided to detect an approaching user by infrared lightreflection, and an ambient light sensor 1070B is provided to sensevisible light. The proximity sensor 1070A can be used to detectproximity in the range of about one meter so that the smart thermostat102 can initiate “waking up” when the user is approaching the thermostatand prior to the user touching the thermostat. Such use of proximitysensing is useful for enhancing the user experience by being “ready” forinteraction as soon as, or very soon after the user is ready to interactwith the thermostat. Further, the wake-up-on-proximity functionalityalso allows for energy savings within the thermostat by “sleeping” whenno user interaction is taking place our about to take place. The ambientlight sensor 1070B can be used for a variety of intelligence-gatheringpurposes, such as for facilitating confirmation of occupancy when sharprising or falling edges are detected (because it is likely that thereare occupants who are turning the lights on and off), and such as fordetecting long term (e.g., 24-hour) patterns of ambient light intensityfor confirming and/or automatically establishing the time of day.

According to some embodiments, for the combined purposes of inspiringuser confidence and further promoting visual and functional elegance,the smart thermostat 102 is controlled by only two types of user input,the first being a rotation of the outer ring 1012 as shown in FIG. 10A(referenced hereafter as a “rotate ring” or “ring rotation” input), andthe second being an inward push on an outer cap 1008 (see FIG. 10B)until an audible and/or tactile “click” occurs (referenced hereafter asan “inward click” or simply “click” input). For the embodiment of FIGS.10A-10B, the outer cap 1008 is an assembly that includes all of theouter ring 1012, cover 1014, electronic display 1016, and metallicportion 1024. When pressed inwardly by the user, the outer cap 1008travels inwardly by a small amount, such as 0.5 mm, against an interiormetallic dome switch (not shown), and then springably travels backoutwardly by that same amount when the inward pressure is released,providing a satisfying tactile “click” sensation to the user's hand,along with a corresponding gentle audible clicking sound. Thus, for theembodiment of FIGS. 10A-10B, an inward click can be achieved by directpressing on the outer ring 1012 itself, or by indirect pressing of theouter ring by virtue of providing inward pressure on the cover 1014,metallic portion 1024, or by various combinations thereof. For otherembodiments, the smart thermostat 102 can be mechanically configuredsuch that only the outer ring 1012 travels inwardly for the inward clickinput, while the cover 1014 and metallic portion 1024 remain motionless.It is to be appreciated that a variety of different selections andcombinations of the particular mechanical elements that will travelinwardly to achieve the “inward click” input are within the scope of thepresent teachings, whether it be the outer ring 1012 itself, some partof the cover 1014, or some combination thereof. However, it has beenfound particularly advantageous to provide the user with an ability toquickly go back and forth between registering “ring rotations” and“inward clicks” with a single hand and with minimal amount of time andeffort involved, and so the ability to provide an inward click directlyby pressing the outer ring 1012 has been found particularlyadvantageous, since the user's fingers do not need to be lifted out ofcontact with the device, or slid along its surface, in order to gobetween ring rotations and inward clicks. Moreover, by virtue of thestrategic placement of the electronic display 1016 centrally inside therotatable ring 1012, a further advantage is provided in that the usercan naturally focus their attention on the electronic display throughoutthe input process, right in the middle of where their hand is performingits functions. The combination of intuitive outer ring rotation,especially as applied to (but not limited to) the changing of athermostat's setpoint temperature, conveniently folded together with thesatisfying physical sensation of inward clicking, together withaccommodating natural focus on the electronic display in the centralmidst of their fingers' activity, adds significantly to an intuitive,seamless, and downright fun user experience. Further descriptions ofadvantageous mechanical user-interfaces and related designs, which areemployed according to some embodiments, can be found in U.S. Ser. No.13/033,573, supra, U.S. Ser. No. 29/386,021, supra, and U.S. Ser. No.13/199,108.

FIG. 10C illustrates a cross-sectional view of a shell portion 1009 of aframe of the thermostat of FIGS. 10A-B, which has been found to providea particularly pleasing and adaptable visual appearance of the overallsmart thermostat 102 when viewed against a variety of different wallcolors and wall textures in a variety of different home environments andhome settings. While the thermostat itself will functionally adapt tothe user's schedule as described herein and in one or more of thecommonly assigned incorporated applications, supra, the outer shellportion 1009 is specially configured to convey a “chameleon” quality orcharacteristic such that the overall device appears to naturally blendin, in a visual and decorative sense, with many of the most common wallcolors and wall textures found in home and business environments, atleast in part because it will appear to assume the surrounding colorsand even textures when viewed from many different angles. The shellportion 1009 has the shape of a frustum that is gently curved whenviewed in cross-section, and comprises a sidewall 1076 that is made of aclear solid material, such as polycarbonate plastic. The sidewall 1076is backpainted with a substantially flat silver- or nickel-coloredpaint, the paint being applied to an inside surface 1078 of the sidewall1076 but not to an outside surface 1077 thereof. The outside surface1077 is smooth and glossy but is not painted. The sidewall 1076 can havea thickness T of about 1.5 mm, a diameter d1 of about 78.8 mm at a firstend that is nearer to the wall when mounted, and a diameter d2 of about81.2 mm at a second end that is farther from the wall when mounted, thediameter change taking place across an outward width dimension “h” ofabout 22.5 mm, the diameter change taking place in either a linearfashion or, more preferably, a slightly nonlinear fashion withincreasing outward distance to form a slightly curved shape when viewedin profile, as shown in FIG. 10C. The outer ring 1012 of outer cap 1008is preferably constructed to match the diameter d2 where disposed nearthe second end of the shell portion 1009 across a modestly sized gap g1therefrom, and then to gently arc back inwardly to meet the cover 1014across a small gap g2. It is to be appreciated, of course, that FIG. 10Conly illustrates the outer shell portion 1009 of the smart thermostat102, and that there are many electronic components internal thereto thatare omitted from FIG. 10C for clarity of presentation, such electroniccomponents being described further hereinbelow and/or in other ones ofthe commonly assigned incorporated applications, such as U.S. Ser. No.13/199,108.

According to some embodiments, the smart thermostat 102 includes aprocessing system 1060, display driver 1064 and a wirelesscommunications system 1066. The processing system 1060 may be disposedwithin a housing of smart thermostat 102, coupled to one or moretemperature sensors of smart thermostat 102 and/or coupled to rotatablering 1012. The processing system 1060 may be configured to dynamicallyidentify user input via rotatable ring 1012, dynamically identifying avariable value (e.g., a setpoint temperature value), and/or dynamicallyidentify an HVAC-control-related property. The processing system 1060may be configured and programmed to provide an interactive thermostatmenuing system (e.g., such as the menuing system shown in FIG. 5) ondisplay area 1016 responsive to an inward pressing of rotatable ring1012 and/or to provide user navigation within the interactive thermostatmenuing system based on rotation of rotatable ring 1012 and inwardpressing of rotatable ring 1012 (e.g., such as is described in relationto FIG. 5). The processing system 1060 may be adapted to cause thedisplay driver 1064 and display area 1016 to display information to theuser and/or to receive user input via the rotatable ring 1012.

For example, an active variable (e.g., variable-value selection,setpoint selection, zip-code selection) may be determined based on adefault state, smart logic or previously received user input. Arelationship between the variable and user input may be identified. Therelationship may be, e.g., linear or non-linear, continuous or discrete,and/or saturating or non-saturating. Such relationships may bepre-defined and stored within the thermostat. User input may bedetected. Analysis of the user input may include, e.g., identifying: atype of user input (tapping versus rotation), a degree of input (e.g., adegree of rotation); a final input position (e.g., a final angularposition of the rotatable ring); an input location (e.g., a position ofa tapping); and/or a speed of input (e.g., a speed of rotation). Usingthe relationship, the processing system 1060 may then determine adisplay indicator, such as a digital numerical value representative ofan identified value of a variable (e.g., a setpoint temperature). Thedisplay indicator may be displayed on display area 1016. For example, adigital numerical value representative of a setpoint temperature to bedisplayed may be determined based on a prior setpoint value and asaturating and continuous relationship between rotation input and thetemperature. The displayed value may be, e.g., numeric, textual orgraphical.

The processing system 1060 may further set a variable value inaccordance with a user selection. For example, a particular type of userinput (e.g., inwards pressure exertion) may be detected. A value of aselected variable may be determined based on, e.g., a prior ringrotation, displayed variable value, etc. The variable may then be set tothis value.

The processing system 1060, according to some embodiments, is capable ofcarrying out the governance of the operation of smart thermostat 102including the user interface features described herein. The processingsystem 1060 is further programmed and configured to carry out otheroperations as described further hereinbelow and/or in other ones of thecommonly assigned incorporated applications. For example, processingsystem 1060 is further programmed and configured to maintain and updatea thermodynamic model for the enclosure in which the HVAC system isinstalled, such as described in U.S. Ser. No. 12/881,463. According tosome embodiments, the wireless communications system 1066 is used tocommunicate with devices such as personal computers and/or otherthermostats or HVAC system components, which can be peer-to-peercommunications, communications through one or more servers located on aprivate network, and/or communications through a cloud-based service.

It should be appreciated that the smart thermostat 102 could be equippedwith a tactile mechanical feedback feature for providingsoftware-triggered mechanical feedback to encourage energy efficientuser behavior. For example, in the event a user attempts to input anill-advised temperature setting (e.g., a setting that consumes a largeamount of energy), the tactile mechanical feedback features provide theuser with negative mechanical feedback, such as by vibrating, making thering hard or impossible to turn.

Further, it should be appreciated that any of the smart devicesdescribed herein, such as the smart hazard detector, the smartthermostat, the smart wall switch, the smart doorbell, could have meansfor self-generating power and optionally storing the generated power ina local battery. For example, in some embodiments, the smart devicesinclude a Peltier Junction for generating power. In these embodiments,for example, the Peltier Junction generates electricity from heatdifferentials created between the smart device and its mountinglocation, such as when the smart device gets hot from use. In otherembodiments, the smart devices are equipped with generators, such aspiezoelectric devices that generate electricity when the device isphysically used, such as when a user turns the ring on the smartthermostat. While piezoelectric devices are used in this example, itshould be appreciated that any generator devices known to those havingskill in the art could be used.

Referring next to FIG. 11, an exemplary environment with whichembodiments may be implemented is shown with a computer system 1100 thatcan be used by a user 1104 to remotely control, for example, one or moreof the sensor-equipped smart-home devices according to one or more ofthe embodiments. The computer system 1110 can alternatively be used forcarrying out one or more of the server-based processing paradigmsdescribed hereinabove, can be used as a processing device in a largerdistributed virtualized computing scheme for carrying out the describedprocessing paradigms, or for any of a variety of other purposesconsistent with the present teachings. The computer system 1100 caninclude a computer 1102, keyboard 1122, a network router 1112, a printer1108, and a monitor 1106. The monitor 1106, processor 1102 and keyboard1122 are part of a computer system 1126, which can be a laptop computer,desktop computer, handheld computer, mainframe computer, etc. Themonitor 1106 can be a CRT, flat screen, etc.

A user 1104 can input commands into the computer 1102 using variousinput devices, such as a mouse, keyboard 1122, track ball, touch screen,etc. If the computer system 1100 comprises a mainframe, a designer 1104can access the computer 1102 using, for example, a terminal or terminalinterface. Additionally, the computer system 1126 may be connected to aprinter 1108 and a server 1110 using a network router 1112, which mayconnect to the Internet 1118 or a WAN.

The server 1110 may, for example, be used to store additional softwareprograms and data. In one embodiment, software implementing the systemsand methods described herein can be stored on a storage medium in theserver 1110. Thus, the software can be run from the storage medium inthe server 1110. In another embodiment, software implementing thesystems and methods described herein can be stored on a storage mediumin the computer 1102. Thus, the software can be run from the storagemedium in the computer system 1126. Therefore, in this embodiment, thesoftware can be used whether or not computer 1102 is connected tonetwork router 1112. Printer 1108 may be connected directly to computer1102, in which case, the computer system 1126 can print whether or notit is connected to network router 1112.

With reference to FIG. 12, an embodiment of a special-purpose computersystem 1200 is shown. For example, one or more of intelligent components116, processing engine 306 and components thereof may be aspecial-purpose computer system 1200. The above methods may beimplemented by computer-program products that direct a computer systemto perform the actions of the above-described methods and components.Each such computer-program product may comprise sets of instructions(codes) embodied on a computer-readable medium that directs theprocessor of a computer system to perform corresponding actions. Theinstructions may be configured to run in sequential order, or inparallel (such as under different processing threads), or in acombination thereof. After loading the computer-program products on ageneral purpose computer system 1126, it is transformed into thespecial-purpose computer system 1200.

Special-purpose computer system 1200 comprises a computer 1102, amonitor 1106 coupled to computer 1102, one or more additional useroutput devices 1230 (optional) coupled to computer 1102, one or moreuser input devices 1240 (e.g., keyboard, mouse, track ball, touchscreen) coupled to computer 1102, an optional communications interface1250 coupled to computer 1102, a computer-program product 1205 stored ina tangible computer-readable memory in computer 1102. Computer-programproduct 1205 directs system 1200 to perform the above-described methods.Computer 1102 may include one or more processors 1260 that communicatewith a number of peripheral devices via a bus subsystem 1290. Theseperipheral devices may include user output device(s) 1230, user inputdevice(s) 1240, communications interface 1250, and a storage subsystem,such as random access memory (RAM) 1270 and non-volatile storage drive1280 (e.g., disk drive, optical drive, solid state drive), which areforms of tangible computer-readable memory.

Computer-program product 1205 may be stored in non-volatile storagedrive 1280 or another computer-readable medium accessible to computer1102 and loaded into memory 1270. Each processor 1260 may comprise amicroprocessor, such as a microprocessor from Intel® or Advanced MicroDevices, Inc.®, or the like. To support computer-program product 1205,the computer 1102 runs an operating system that handles thecommunications of product 1205 with the above-noted components, as wellas the communications between the above-noted components in support ofthe computer-program product 1205. Exemplary operating systems includeWindows®. or the like from Microsoft Corporation, Solaris® from SunMicrosystems, LINUX, UNIX, and the like.

User input devices 1240 include all possible types of devices andmechanisms to input information to computer system 1102. These mayinclude a keyboard, a keypad, a mouse, a scanner, a digital drawing pad,a touch screen incorporated into the display, audio input devices suchas voice recognition systems, microphones, and other types of inputdevices. In various embodiments, user input devices 1240 are typicallyembodied as a computer mouse, a trackball, a track pad, a joystick,wireless remote, a drawing tablet, a voice command system. User inputdevices 1240 typically allow a user to select objects, icons, text andthe like that appear on the monitor 1106 via a command such as a clickof a button or the like. User output devices 1230 include all possibletypes of devices and mechanisms to output information from computer1102. These may include a display (e.g., monitor 1106), printers,non-visual displays such as audio output devices, etc.

Communications interface 1250 provides an interface to othercommunication networks and devices and may serve as an interface toreceive data from and transmit data to other systems, WANs and/or theInternet 1118. Embodiments of communications interface 1250 typicallyinclude an Ethernet card, a modem (telephone, satellite, cable, ISDN), a(asynchronous) digital subscriber line (DSL) unit, a FireWire®interface, a USB® interface, a wireless network adapter, and the like.For example, communications interface 1250 may be coupled to a computernetwork, to a FireWire® bus, or the like. In other embodiments,communications interface 1250 may be physically integrated on themotherboard of computer 1102, and/or may be a software program, or thelike.

RAM 1270 and non-volatile storage drive 1280 are examples of tangiblecomputer-readable media configured to store data such ascomputer-program product embodiments of the present invention, includingexecutable computer code, human-readable code, or the like. Other typesof tangible computer-readable media include floppy disks, removable harddisks, optical storage media such as CD-ROMs, DVDs, bar codes,semiconductor memories such as flash memories, read-only-memories(ROMs), battery-backed volatile memories, networked storage devices, andthe like. RAM 1270 and non-volatile storage drive 1280 may be configuredto store the basic programming and data constructs that provide thefunctionality of various embodiments of the present invention, asdescribed above.

Software instruction sets that provide the functionality of the presentinvention may be stored in RAM 1270 and non-volatile storage drive 1280.These instruction sets or code may be executed by the processor(s) 1260.RAM 1270 and non-volatile storage drive 1280 may also provide arepository to store data and data structures used in accordance with thepresent invention. RAM 1270 and non-volatile storage drive 1280 mayinclude a number of memories including a main random access memory (RAM)to store of instructions and data during program execution and aread-only memory (ROM) in which fixed instructions are stored. RAM 1270and non-volatile storage drive 1280 may include a file storage subsystemproviding persistent (non-volatile) storage of program and/or datafiles. RAM 1270 and non-volatile storage drive 1280 may also includeremovable storage systems, such as removable flash memory.

Bus subsystem 1290 provides a mechanism to allow the various componentsand subsystems of computer 1102 communicate with each other as intended.Although bus subsystem 1290 is shown schematically as a single bus,alternative embodiments of the bus subsystem may utilize multiple bussesor communication paths within the computer 1102.

For a firmware and/or software implementation, the methodologies may beimplemented with modules (e.g., procedures, functions, and so on) thatperform the functions described herein. Any machine-readable mediumtangibly embodying instructions may be used in implementing themethodologies described herein. For example, software codes may bestored in a memory. Memory may be implemented within the processor orexternal to the processor. As used herein the term “memory” refers toany type of long term, short term, volatile, nonvolatile, or otherstorage medium and is not to be limited to any particular type of memoryor number of memories, or type of media upon which memory is stored.

Moreover, as disclosed herein, the term “storage medium” may representone or more memories for storing data, including read only memory (ROM),random access memory (RAM), magnetic RAM, core memory, magnetic diskstorage mediums, optical storage mediums, flash memory devices and/orother machine readable mediums for storing information. The term“machine-readable medium” includes, but is not limited to portable orfixed storage devices, optical storage devices, wireless channels,and/or various other storage mediums capable of storing that contain orcarry instruction(s) and/or data.

Referring now to FIG. 13, provided according to some embodiments aresystems and methods for achieving smart-home objectives usingoccupant-location data (e.g., GPS data). According to embodiments,occupant-location data—in addition to other data obtained from smarthome devices—is provided to the central server or cloud-computing system164, which uses the data to make inferences regarding the current andfuture occupancy of the home and/or rooms, and to control the smartdevices inside the home in a corresponding manner. Further, according toembodiments, provided are systems and methods for conflict sensing andresolution for accommodating multiple occupants in the smart-homeenvironment 100, which has occupant-location based controls of the smarthome devices, such as temperature adjustments via the smart thermostat.For example, as described below, the systems and methods identifyconflicts in the preferences (e.g., preferred temperature settings) ofthe occupants of the home and resolves the conflicts based on prescribedor learned rules, such has giving priority to the preferences of someoccupants.

According to embodiments, the central server or cloud-computing system164 obtains occupant-location data, such as GPS data, from the mobiledevices 166 of the occupants. For example, the occupants of a home(e.g., the individuals who live in or frequently visit the home)register their respective mobile devices 166 as being associated withthe home.

In some instances the central server or cloud-computing system 164receives occupant-location data directly from the mobile devices,whereas in others the data is received from an intermediary, such as oneof the smart devices in the home.

In instances where occupant-location data is received directly from themobile device, the central server or cloud-computing system 164 candetermine if the occupant is “at home” or “away”, as explained below.When an occupant is at home, the central server or cloud-computingsystem 164 can determine the occupant's actual room-location (e.g.,bedroom, kitchen, garage, etc.). To do, for example, the central serveror cloud-computing system 164 cross-references the occupant-location(e.g., GPS coordinates) with a map of the home.

In instances where occupant-location data is received from smartdevices, the central server or cloud-computing system 164 can infer thatthe occupant is inside the home. Further, the room-location of theoccupants can be determined. For example, the smart wall switches, thesmart wall plugs, the smart doorbells, and other smart devices in thesmart-home environment 100 detect the presence of the mobile device 166and transmit corresponding data to the central server or cloud-computingsystem 164. Such detection of mobile devices can be accomplished usingWiFi, Bluetooth, NFC, etc. It should also be appreciated that passiveRFID tags can be used to determine the room-location of occupants (andpets). For example, an RFID is associated with each of the occupants(and pets) of the house, such as by including the tags in wallets,bracelets, wristbands, mobile devices, collars, etc. The smart devicesin the various rooms detect the RFID tags, and send that information tothe central server or cloud-computing system 164. It should beappreciated that, because they are typical mounted in unobstructedlocations, high on walls of often-occupied rooms, smart hazard detectors104 are particularly well suited for RFID sensors.

In the illustrated example, the occupants include Wife 1314, Husband1318 and Child 1322, all of whom have registered their mobile device 166with the central server or cloud-computing system 164 as beingassociated with the smart-home environment 100. Further, twogeo-location boundaries or “geo-fences” 1330, 1334 are registered asbeing associated with the smart-home environment 100. In someembodiments, the occupants of the home define and register thegeo-fences, while in other embodiments the central server orcloud-computing system 164 auto-generates the geo-fences for the home.

Inner geo-fence 1330 defines the perimeter of living area of the home.The area within the inner-geo fence includes not only the home but alsothe land immediately surrounding the house, including any closelyassociated structures, such as garages or sheds (“the cartilage”). Outergeo-fence 1334 defines an outer perimeter, which is sometimes miles fromthe home. The outer geo-fence 1334 is adjustable and extends well beyondthe curtilage. For example, the perimeter defined by the outer geo-fence1334 may have a radius of two to three miles in some embodiments, whilein other embodiments the radius is larger or smaller.

According to embodiments, the central server or cloud-computing system164 infers that an occupant is “at home” when inside the inner geo-fence1330 and that the occupant is “away” when outside of the inner geo-fence1330. Further, the central server or cloud-computing system 164 infersan occupant is going home when the occupant moves (e.g., travels by car)from outside to inside the outer geo-fence 1334. As such, the centralserver or cloud-computing system 164 uses the inner geo-fence 1330 todetermine when occupants leave the home, and it uses the outer geo-fence1334 to determine when occupants are heading home.

Several exemplary smart-home objectives will now be described withreference to FIG. 13. In one example, the central server orcloud-computing system 164 predicts that Wife 1314 is going home, andmakes adjustments in the smart-home environment 100 in anticipation ofher arrival. In this example, Wife 1314 travels from outside to insidethe outer geo-fence 1334, thereby triggering an indication that Wife isheading home. The Wife's speed and the distance between the fence andthe home 100 is used to predict the Wife's time of arrival. Using thisinformation central server or cloud-computing system 164 can prepare forWife's arrival, such as by pre-heating or cooling some or all of therooms in the home 100 to Wife's preferred temperature.Time-to-temperature calculations can be used to make the pre-cooling orheading as efficient as possible, and to ensure Wife's preferredtemperature is achieved before she arrives. In the event neither Husband1318 nor Child 1322 is home, all rooms are pre-heated or cooled.However, if either or both Husband 1318 and Child 1322 are home, roomsassociate with Wife can be pre-heated or cooled to Wife's desiredtemperature, while other rooms are set to temperatures based thepreferences of Husband 1318 and/or Child 1322. Rooms associated withWife can be inferred by the central server or cloud-computing system 164based on historical occupancy and usage data, as well as based on a setof rules provided by the occupants (see below).

In another example, the central server or cloud-computing system 164determines that the home 100 is vacant because all occupants (Wife 1314,Husband 1318, and Child 1322) are outside of the inner geo-fence 1330.In this case, the central server or cloud-computing system 164 placesthe smart devices in their “away” settings. For example, the lights areturned off, the HVAC is set to a temperature that requires little or noheating or cooling, the alarm is armed, the doors are locked,electronics and appliances (e.g., televisions, stoves, space heaters,etc.) are turned off, etc. In the event, all occupants' mobile devicesare outside of the inner geo-fence 1330 and movement is detected in thehouse, the central server or cloud-computing system 164 can infer themovement is the family dog and can apply the dog's smart devicessettings.

In yet another example, Wife 1314, Husband 1318, and Child 1322 are allinside the inner geo-fence 1320. Applying the techniques describedabove, the central server or cloud-computing system 164 knows that Wife1314 is in the master bedroom and that Husband 1318 and Child 1322 areboth in the den (see discussion regarding “room-occupancy detection”).Accordingly, the central server or cloud-computing system 164 adjuststhe temperature in the master bedroom to Wife's preferred setting, andit applies a set of rules to determine whether to set the den to theHusband's or the Child's preferred setting. The set of rule, for eachroom of the home, provide a prioritized list of occupants and theirrespective settings. For example, for the den, Wife's settings arehighest priority, then Husband's, and then the Child's. Thus, in thiscase, Husband's setting are applied in the den because his setting arehigher priority than Child's. However, if Wife were to later join themin the den, her setting would be applied instead of Husband's.

In still another example, Wife 1314, Husband 1318, and Child 1322 areall inside the inner geo-fence 1320. Wife and Husband are in the den,but Child is in the yard playing. The central server or cloud-computingsystem 164 applies the above-described techniques to determine that Wifeand Husband are in the den. However, Child is unaccounted for in any ofthe rooms. In this example, Child was detected in the den earlier thatday, and none of the occupants have traveled outside of the inner-fence1334 that day. Using this information, the central server orcloud-computing system 164 infers that Child is in the yard, playing.Alternatively, the central server or cloud-computing system 164 infersthat Child is in the yard because it received data from the smartdoorbell that the child exited the house, yet it is also receiving datafrom Child's mobile device that Child is inside of the inner geo-fence1330. Because the Child is in the yard, the central server orcloud-computing system 164 prevents the irrigation system from turningon and it broadcasts occasional reminders to Husband and Wife viaspeakers provided in the smart devices, for example.

In some embodiments, a system is provided for accomplishing an objectivein a smart-home environment. This system comprises one or morelow-powered nodes located in the smart-home environment, and one or moresmart nodes located in the smart-home environment, where the low-poweredand smart nodes communicatively interconnected for the purpose ofaccomplishing the objective in the smart-home environment. In someembodiments, at least one of the low-powered and smart nodes isconfigured to monitor a condition of the smart-home environment, andtransmit to at least one of the other low-powered and smart nodes amessage having information related to the condition of the smart-homeenvironment.

In some embodiments, responsive to receiving a message havinginformation related to a condition of the smart-home environment, atleast one of the smart nodes is configured to determine an objectivethat is to be accomplished in the smart-home environment and that isappropriate in light of the condition, determine a function to beperformed in the smart-home environment for the purpose of accomplishingthe objective, and transmit to the other low-powered and smart nodes ofthe smart-home environment instructions to perform the function in thesmart-home environment for the purpose of accomplishing the objective.In some embodiments, responsive to receiving a message havinginformation related to a condition of the smart-home environment, atleast one of the smart nodes can be configured to transmit to a server amessage having information related to a condition of the smart-homeenvironment. In some embodiments, responsive to receiving a messagehaving information related to the condition of the smart-homeenvironment, the server is configured to determine an objective that isto be accomplished in the smart-home environment and that is appropriatein light of the condition, determine a function to be performed in thesmart-home environment for the purpose of accomplishing the objective,and transmit to at least one of the smart nodes of the smart-homeenvironment instructions to perform the function in the smart-homeenvironment for the purpose of accomplishing the objective.

In some embodiments, the low-powered nodes are capable of communicatingusing a first wireless protocol characterized by relatively low powerusage and relatively low data rates, and the smart nodes are capable ofcommunicating using the first wireless protocol and a second wirelessprotocol characterized by relatively higher power usage and relativelyhigher data rates. In some embodiments, when transmitting to a server amessage having information related to a condition of the smart-homeenvironment, at least one of the smart nodes serves as a communicationbridge to a wide area network using the second wireless protocolcharacterized by relatively higher power usage and relatively higherdata rates.

In some embodiments, the smart nodes are smart devices, such as a hazarddetector unit, a doorbell, a thermostat, a wall switch, or a wall plug.In some embodiments, the low-powered nodes are smart devices. In someembodiments, the smart the smart devices are identical and capable ofperforming as the low-powered node and the smart node. In someembodiments, the condition is a temperature, an amount of light, asound, a movement, a vibration, a smell, a toxin, or an amount of heat.In some embodiments, the function and corresponding objective aretriggering an alarm to secure the smart-home environment, adjusting athermostat setting to make the smart-home environment comfortable, orturning on or off a light to secure the smart-home environment or foruse by occupants of the smart-home environment.

In some embodiments, a method is provided that comprises monitoring, byone or more communicatively interconnected low-powered and smart nodes,a condition of a smart-home environment; transmitting, by at least oneof the low-powered and smart nodes, a message having information relatedto the condition of the smart-home environment; receiving, by at leastone of the smart nodes, a message having information related to thecondition in the smart-home environment; determining, by at least one ofthe smart nodes based at least in part on the information related to thecondition, a function to be performed in the smart-home environment; andtransmitting, by at least one of the smart nodes, a message that causesat least one of the low-powered and smart nodes to perform the functionin the smart-home environment. In some embodiments, the low-powerednodes are capable of communicating using a first wireless protocolcharacterized by relatively low power usage and relatively low datarates, and the smart nodes are capable of communicating using the firstwireless protocol and a second wireless protocol characterized byrelatively higher power usage and relatively higher data rates.

In some embodiments, the method further comprises transmitting, by atleast one of the low-powered and smart nodes, a message havinginformation related to the condition of the smart-home environmentinvolves “repeating” the message to the other low-powered and smartnodes in the smart-home environment. In some embodiments, the smartnodes are smart devices, such as a hazard detector unit, a doorbell, athermostat, a wall switch, or a wall plug. In some embodiments, thelow-powered nodes are smart devices. In some embodiments, the smartdevices are identical and capable of performing as the low-powered nodeand the smart node.

In some embodiments, a home automation system is provided that comprisesa home security system including one or more sensor nodes capable ofdetecting motion within a home, and a service robot system including atleast one service robot that autonomously moves within the home toperform one or more functions, where the home security system and theservice robot system are mutually configured such that there will not bea motion alarm set off by the robot when the robot moves within a rangeof the one or more sensor nodes. In some embodiments, the system furthercomprises a computing device provided in operative communication withthe home security system and the service robot system, where thecomputing device configured to receive in-home location coordinates fromthe robot, and distinguish between activity associated with the robotand unexpected intrusion activity by using the in-home locationcoordinates to filter signals from the one more sensor nodes. In someembodiments, the computing device is an on-site computing device. Insome embodiments, the computing device is a remote server.

In some embodiments, the robot outputs a signal when moving to alert thehome security system to its presence so that there will not be a motionalarm set off for as long as the signal is detected by the one or moresensor nodes. In some embodiments, the signal is authenticated andencrypted such that the signal cannot be learned and replicated by apotential burglar. In some embodiments, signal authentication andencryption is accomplished by a permission request-response scheme,where the service robot system requests permission from home securitysystem when the service robot system is ready to deploy the robot toperform the one or more functions. In some embodiments, the servicerobot system does not deploy the robot until receiving a “yes” messagefrom home security system. In some embodiments, the signal is one ormore of an optical notifying signal, an audible notifying signal, aninfrared notifying signal, an infrasonic notifying signal, or awirelessly transmitted data notification signal. In some embodiments,the one or more functions performed by the robot include one or more offloor sweeping, floor washing, playing music for an occupant, serving asa localized thermostat for an occupant, serving as a localized airmonitor/purifier for an occupant, serving as a localized baby monitor,or serving as a localized hazard detector for an occupant.

In some embodiments, a wall switch controller is provided that comprisesa docking station configured to receive a user-removable wall-switchhead unit, the docking station configured to be permanently connected toa wall and coupled to high-power voltage wires, and a user-removablewall-switch head unit configured to be user-insertable into said dockingstation and user-removable therefrom such that the user is not exposedto high-voltage connections when inserting or removing. In someembodiments, the wall switch controller further comprises inputs andoutputs and circuitry for switchably controlling household line currentpower to at least one household electrical fixture. In some embodiments,the wall switch controller further comprises one or more of an occupancysensor, a temperature sensor, or a processor. In some embodiments, thewall switch controller further comprises circuitry for leveragingelectrical power from the household line current power inputs to powerone or more of the occupancy sensor, the temperature sensor, or theprocessor.

In some embodiments, the wall switch controller further compriseswireless communication circuitry configured to communicate with aplurality of low-powered devices using a first wireless protocolcharacterized by relatively low power usage and relatively lower datarates and to serve as a communications bridge to a wide area networkusing a second wireless protocol characterized by relatively higherpower usage and relatively higher data rates, and a power storage modulefor storing power and using the power to maintain wirelesscommunications during a power outage. In some embodiments, the wallswitch controller further comprises a housing configured to becompatible for installation in a conventional single-gang ormultiple-gang wall box.

In some embodiments, a method is provided for providing an alarm clockin a smart-home environment, and the method comprises obtaining, at aserver, a wake time for an occupant of the smart-home environment, andinstructing, by the server, one or more smart devices in the smart-homeenvironment to output an audible alarm when the wake time occurs. Insome embodiments, obtaining a wake time comprises obtaining over aperiod, from one or more motion-sensing smart devices located in thesmart-home environment, data related to movement of the occupant inresponse to an alarm, and inferring, by the server, a wake time based atleast in part on the data related to movement of the occupant inresponse to an alarm over the period. In some embodiments, the methodfurther comprises tracking, by the one or more motion-detecting smartdevices, movement of the occupant between one or more locations withinthe smart-home environment; detecting when the occupant stops moving fora period; and inferring that the occupant has fallen asleep at alocation within the smart-home environment.

In some embodiments, the method further comprises inferring where andwhen the occupant prefers to sleep. In some embodiments, instructing oneor more smart devices in the smart-home environment to output an audiblealarm when the wake time occurs, comprises instructing only the one ofthe one or more smart devices that is closest to the occupant to outputthe audible alarm. In some embodiments, tracking a location of theoccupant within the smart-home environment, comprises generating aunique signature for the occupant based at least in part of dataobtained from the one or more smart devices, and using the uniquesignature to distinguish the occupant from other occupants of thesmart-home environment. In some embodiments, the unique signature of theoccupant is based on one or more of a walking gate, patterns ofmovement, voice, height, or size. In some embodiments, the data that isobtained from the one or more smart devices and that is used to generatethe unique signature of obtained from at least one of an ultrasonicsensor and a passive IR sensor included in the one or moremotion-detecting smart devices.

In some embodiments, the method further comprises transmitting, to athermostat, instructions to heat or cool the smart-home environment to adesired “sleeping” temperature setting when the occupant is determinedto be sleeping. In some embodiments, the method further comprisestransmitting, to a thermostat, instructions to heat or cool thesmart-home environment to a desired “awake” temperature setting when theoccupant is not determined to be sleeping. In some embodiments, thedesired “sleeping” and “awake” temperature settings can be learned overtime, such as be observing which temperature the occupant sets thethermostat to before going to sleep and which temperature the occupantsets the thermostat to upon waking up. In some embodiments, the methodfurther comprises transmitting, to a thermostat, instructions topre-heat or pre-cool the smart-home environment to a desired “awake”temperature setting in anticipation of the occupant waking up.

In some embodiments a method is provided that comprises obtaining, at aserver from one or more smart devices located in one or more smart-homeenvironments, home data collected from the smart-home environments byone or more sensors of the smart devices; and exposing, by the server,one or more application program interfaces for access by one or moreservice-provider entities to obtain the home data for the purpose ofoptimizing one or more home services provided by the one or moreservice-provider entities to the one or more smart-home environments. Insome embodiments, the data includes at least one of smart device usagepatterns and occupant presence patterns. In some embodiments, the serverderives aggregate patterns from the home data. In some embodiments, theserver provides neighborhood-level probabilities regarding whetheroccupants may be currently present in the smart-home environments. Insome embodiments, the one or more home services being optimized aredelivery services whose effectiveness is improved when the occupants areat home. In some embodiments, the application program interfacesanonymize the home data such that any one of the smart-home environmentsis not identifiable. In some embodiments, the one or more home servicesbeing optimized are not functionally related to the home data beingcollected by the one or more smart sensors.

In some embodiments a home monitoring and control system is providedthat comprises a plurality of hazard detectors with wireless capabilityand occupancy detectors and audio speakers, and a doorbell having aprocessor and wireless capability, where a doorbell-ringing eventtriggers the processor to cause the doorbell to wirelessly communicatewith one or more of the hazard detectors to cause the audio speakers toalert occupants of a visitor at the door. In some embodiments, eachhazard detector can be configured with a do-not-disturb setting suchthat it will remain silent under one or more pre-determined conditions.In some embodiments, the pre-determined conditions include one or moreof an occupant determined to be sleeping proximate to the hazarddetector, room location of hazard detector is unoccupied, hazarddetector is located in an occupied bedroom; or hazard detector islocated in a kid's bedroom.

What is claimed is:
 1. A home monitoring and control system, comprising:a doorbell installed at a door of a home including: a button configuredto, upon being touched, depressed or activated, wirelessly initiate afirst communication to cause an audio speaker of an electronic device tosound an audible notification and/or a message to be sent to amonitoring device associated with an occupant of the home to indicatepresence of a person at the door; a camera configured to capture videodata within a field of view; and a processor and a communicationcomponent, wherein the processor is configured to cause thecommunication component to wirelessly stream via a remote server thevideo data captured by the camera to the monitoring device; and anapplication executable by the monitoring device, the application beingconfigured to, when executed by the monitoring device, receive from thedoorbell via the remote server at least a portion of the video datacaptured by the camera of the doorbell, and display the received videodata captured by the camera of the doorbell; wherein the audio speakeris distinct from the doorbell, and is configured to communicatewirelessly with the doorbell to sound the audible notification inresponse to the first communication from the doorbell; wherein theremote server is located remotely from the doorbell, the monitoringdevice and the audio speaker, and the remote server is configured to:receive from the doorbell via the communication component the video datacaptured by the camera and streamed to the remote server; receive themessage indicating the presence of the person at the door; store thereceived video data; and relay the received video data and the messageto the monitoring device; wherein the audio speaker of the electronicdevice has a do-not-disturb setting defining one or more pre-determinedconditions, and in accordance with the do-no-disturb setting, the audiospeaker of the electronic device is configured to remain silent underthe one or more pre-determined conditions.
 2. The home monitoring andcontrol system of claim 1, wherein: the electronic device includes ahazard detector; the first hazard detector includes the audio speakerand has wireless capability; and the first hazard detector is wirelesslycoupled to the doorbell via one or more wireless networks.
 3. The homemonitoring and control system of claim 1, wherein the electronic deviceincludes a first electronic device, further comprising: a secondelectronic device, wherein the second electronic device includes asecond audio speaker and has wireless capability, and is configured tocommunicate wirelessly with the doorbell to sound a second audiblenotification in response to the doorbell being touched, depressed oractivated, concurrently with the sounding of the audible notification bythe audio speaker of the first electronic device.
 4. The home monitoringand control system of claim 3, wherein the first electronic device is afirst hazard detector, and the second electronic device is a secondhazard detector distinct from the first hazard detector.
 5. The homemonitoring and control system of claim 1, wherein the monitoring deviceincludes a wall switch, and the wall switch is configured to display thevideo data captured by the camera on a surface of the wall switch orproject the video data in a 2D or 3D form.
 6. The home monitoring andcontrol system of claim 1, wherein the doorbell is configured to receiveair quality information from a smart hazard detector via the remoteserver.
 7. The home monitoring and control system of claim 1, whereinthe doorbell further includes a speaker, and is configured to: inaccordance with a determination that the person is within apredetermined distance from the door, broadcast via the speaker aquestion asking whether the person is a delivery person; receive aresponse from the person via a microphone of the doorbell; and inaccordance with a determination that the person is a delivery personbased on the response, present a user instruction to instruct the personto place a package to be delivered in a location proximate to thedoorbell.
 8. The home monitoring and control system of claim 7, whereinthe doorbell further includes a scanner configured to scan a bar code ora tag identifying the package being delivered by the person.
 9. The homemonitoring and control system of claim 1, wherein the doorbell furtherincludes a speaker, and is configured to: in response to a detection viathe remote server that an unknown individual is approaching the home,broadcast to the unknown individual via the speaker an audible notice.10. The home monitoring and control system of claim 1, wherein themonitoring device includes a mobile device associated with the occupantof the home.
 11. The home monitoring and control system of claim 1,wherein the broadcasting of the audible notification is aborted inaccordance with a determination that a baby is sleeping in a room wherethe audio speaker is located.
 12. The home monitoring and control systemof claim 1, wherein the doorbell includes an RFID scanner configured toscan a mobile phone of the person, and the monitoring device isconfigured to identify the person based on information collected by theRFID scanner and receive a user instruction to grant an access right tothe person based on the information collected by the RFID scanner.
 13. Ahome monitoring and control system, comprising: a doorbell installed ata door of a home including: a button configured to, upon being touched,depressed or activated, wirelessly initiate a first communication tocause an audio speaker of an electronic device to sound an audiblenotification and/or a message to be sent to a monitoring deviceassociated with an occupant of the home to indicate presence of a personat the door; a camera configured to capture video data within a field ofview; and a processor and a communication component, wherein theprocessor is configured to cause the communication component towirelessly stream via a remote server the video data captured by thecamera to the monitoring device; and an application executable by themonitoring device, the application being configured to, when executed bythe monitoring device, receive from the doorbell via the remote serverat least a portion of the video data captured by the camera of thedoorbell, and display the received video data captured by the camera ofthe doorbell; wherein the audio speaker is distinct from the doorbell,and is configured to communicate wirelessly with the doorbell to soundthe audible notification in response to the first communication from thedoorbell; wherein the remote server is located remotely from thedoorbell, the monitoring device and the audio speaker, and the remoteserver is configured to: receive from the doorbell via the communicationcomponent the video data captured by the camera and streamed to theremote server; receive the message indicating the presence of the personat the door; store the received video data; and relay the received videodata and the message to the monitoring device; wherein the doorbell isconfigured to detect a visitor activity including whether the person isapproaching or leaving the door and generate a security control signalto control settings of a security system according to the detectedvisitor activity.
 14. The home monitoring and control system of claim13, wherein the audio speaker of the electronic device has ado-not-disturb setting defining one or more pre-determined conditions,and in accordance with the do-no-disturb setting, the audio speaker ofthe electronic device is configured to remain silent under the one ormore pre-determined conditions.
 15. The home monitoring and controlsystem of claim 14, wherein the one or more pre-determined conditionsinclude one or more of: the occupant is determined to be sleepingproximate to the audio speaker; a room where the audio speaker of theelectronic device is located is unoccupied; the audio speaker of theelectronic device is located in an occupied bedroom; and the audiospeaker of the electronic device is located in a kid's bedroom.
 16. Thehome monitoring and control system of claim 13, wherein the doorbell isconfigured to in accordance with the settings of the security system:activate the security system when the person is approaching the door;and deactivate the security system when the person is leaving the door.17. A home monitoring and control system, comprising: a doorbellinstalled at a door of a home including: a button configured to, uponbeing touched, depressed or activated, wirelessly initiate a firstcommunication to cause an audio speaker of an electronic device to soundan audible notification and/or a message to be sent to a monitoringdevice associated with an occupant of the home to indicate presence of aperson at the door; a camera configured to capture video data within afield of view; a rechargeable battery electrically coupled to a housingwire and configured to be charged via the housing wire; and a processorand a communication component, wherein the processor is configured tocause the communication component to wirelessly stream via a remoteserver the video data captured by the camera to the monitoring device;and an application executable by the monitoring device, the applicationbeing configured to, when executed by the monitoring device, receivefrom the doorbell via the remote server at least a portion of the videodata captured by the camera of the doorbell, and display the receivedvideo data captured by the camera of the doorbell; wherein the audiospeaker is distinct from the doorbell, and is configured to communicatewirelessly with the doorbell to sound the audible notification inresponse to the first communication from the doorbell; wherein theremote server is located remotely from the doorbell, the monitoringdevice and the audio speaker, and the remote server is configured to:receive from the doorbell via the communication component the video datacaptured by the camera and streamed to the remote server; receive themessage indicating the presence of the person at the door; store thereceived video data; and relay the received video data and the messageto the monitoring device; wherein the doorbell is configured to chargethe rechargeable battery during first time intervals, and therechargeable battery is configured to discharge to power and enableoperations of the doorbell during second time intervals, the first andsecond time intervals being determined based on power usage of thedoorbell.
 18. The home monitoring and control system of claim 17,wherein the doorbell is configured to charge the rechargeable batteryduring the first time intervals in which the power usage of the doorbellis less than a safe power level provided by the housing wire, and therechargeable battery is configured to discharge to power and enableoperations of the doorbell during second time intervals in which thepower usage of the doorbell is greater than the safe power levelprovided by the housing wire.
 19. A home monitoring and control system,comprising: a doorbell installed at a door of a home including: a buttonconfigured to, upon being touched, depressed or activated, wirelesslyinitiate a first communication to cause an audio speaker of anelectronic device to sound an audible notification and/or a message tobe sent to a monitoring device associated with an occupant of the hometo indicate presence of a person at the door; a camera configured tocapture video data within a field of view; and a processor and acommunication component, wherein the processor is configured to causethe communication component to wirelessly stream via a remote server thevideo data captured by the camera to the monitoring device; and anapplication executable by the monitoring device, the application beingconfigured to, when executed by the monitoring device, receive from thedoorbell via the remote server at least a portion of the video datacaptured by the camera of the doorbell, and display the received videodata captured by the camera of the doorbell; wherein the audio speakeris distinct from the doorbell, and is configured to communicatewirelessly with the doorbell to sound the audible notification inresponse to the first communication from the doorbell; wherein theremote server is located remotely from the doorbell, the monitoringdevice and the audio speaker, and the remote server is configured to:receive from the doorbell via the communication component the video datacaptured by the camera and streamed to the remote server; receive themessage indicating the presence of the person at the door; store thereceived video data; and relay the received video data and the messageto the monitoring device; wherein the remote server is configured todetermine (1) whether a street adjacent to the home has a thresholdlevel of traffic and (2) a safety level for children residing at thehome based on the video data, and generate an alarm based on thedetermination.
 20. The home monitoring and control system of claim 19,wherein the remote server is configured to in accordance with adetermination that the street adjacent to the home has the thresholdlevel of traffic, send an instruction to the electronic device toincrease a volume level of the audio speaker.